Hey lichslap,
You may want to try disabling the DMZ or the port forwarding. Since they are both methods of configuring your ports, they may be interfering with each other. Did you set up a static ip address for your console? Have you tried power cycling your routers? Can you please provide your Speedtest results? If you are going to keep the port forwarding, can you also try these numbers:
Xbox Ports:
53, 80, 3074 TCP
53, 88, 500, 3074, 3544, 4500 UDP
Call of Duty Ghosts Console Ports:
80, 81,443, 3074, 5223 TCP
36081, 3074, 3075, 3478 UDP.
Thanks,
^AJ