Create an Account
username: password:
 
  MemeStreams Logo

MemeStreams Discussion

search


This page contains all of the posts and discussion on MemeStreams referencing the following web page: Comcast's DHCP is broken. You can find discussions on MemeStreams as you surf the web, even if you aren't a MemeStreams member, using the Threads Bookmarklet.

Comcast's DHCP is broken
by Decius at 6:23 pm EST, Dec 15, 2008

I just wasted an entire day trying to inform Comcast about a problem with their network. This problem caused a service outage for me, and I'm sure that it has effected hundreds of other customers. Unfortunately, it will never be fixed.

Senior engineers at Comcast are not aware of the problem, and they will never be aware, because it is impossible to inform them. Comcast's first and second level support staff don't understand the problem and have been trained to aggressively blow off anyone who attempts to report it to them. As far as I can tell, there is absolutely no way to get through the first and second level support barrier to someone who actually understands DHCP. Comcast's support staff does not know when to escalate something that they do not understand.

I am posting this mostly as a personal catharsis having spent an entire day being told that I don't know what I'm talking about by people who barely know the first thing about how the Internet works. Pushing this further is not worth the frustration. Perhaps someone else who is experiencing the same problem will come upon this blog post in a Google search and will be saved the same frustration. That is the only thing that I can do at this point.

The problem manifests as follows: Some devices are intermittently unable to obtain a DHCP lease. What makes this complicated is that other devices ARE still able to obtain a lease. In my case my router stopped getting IPs from Comcast, but I could get an IP with my laptop. The router had been working fine as my gateway for months and had no problems getting IPs and then one day I woke up and it wasn't working anymore. My router could not get a lease, but my laptop could get a lease if I plugged it into my cable modem directly.

The naive assumption when confronted with this set of circumstances is that the problem is with the router. The network is obviously able to hand a lease out. The router must just not be asking for one properly or accepting one as it should. The first time I encountered this behavior on Comcast's network, I bought into this assumption and went out and purchased a new router. Then, a few months later, it happened to my new router as well.

In this case, the naive assumption is wrong; both routers are working properly. Internet protocols are complicated and sometimes they fail in subtle ways that defy naive assumptions. Unfortunately, it is impossible to get Comcast to look at this problem more carefully, because their low level technical support staff don't understand how to look at it more carefully, and believe that the naive assumption is the only possible explanation.

Because Comcast's network has this problem, people likely call up technical support on a regular basis complaining about it, and they are told that their routers must be broken. Comcast's technical support staff has gotten good at arguing people down about their "broken routers" because they see it all time. Of co... [ Read More (0.7k in body) ]


 
RE: Comcast's DHCP is broken
by Tsudo at 11:03 pm EST, Dec 15, 2008

I floated your article to the Comcast guy on Twitter and not surprisingly he said the problem lies with the modem. http://tr.im/2bs8

Quoted from him, "That is what happens when a modem is not connected to the network."

I don't but that at all. It doesn't begin to explain why the Modem ACK. Good luck trying to get further.

Keep us in the loop.


  
RE: Comcast's DHCP is broken
by Decius at 8:47 am EST, Dec 16, 2008

For the record, my current hypothesis is this:

If Comcast's DHCP server thinks my MAC address has a lease from it and I start sending out discoveries that are asking for 192.168.100.10, those discoveries are ignored. They shouldn't be, because Comcast's Cable Modems hand out that address, and gateway devices might obtain it during a power outage that is shorter than the leases that are handed out by the real DHCP server.

I could be wrong about this. Its just a hypothesis based on the information that I have and I don't have any way to test it. But its pretty reasonable.

Getting Comcast to investigate it is impossible. I can't get them to accept the facts that I can prove with packet captures, so getting them to consider a hypothesis... it ain't gunna happen.


 
RE: Comcast's DHCP is broken
by Tsudo at 11:19 pm EST, Dec 15, 2008

One final reply after I asked Comcast about the strange Modem ACK

Referred to it as a Block Sync Error. http://tr.im/2bsy

What do you think?


  
RE: Comcast's DHCP is broken
by Decius at 8:24 am EST, Dec 16, 2008

Tsudohnimh wrote:
I floated your article to the Comcast guy on Twitter and not surprisingly he said the problem lies with the modem. http://tr.im/2bs8

Quoted from him, "That is what happens when a modem is not connected to the network."

I don't but that at all. It doesn't begin to explain why the Modem ACK. Good luck trying to get further.

Keep us in the loop.

Thanks for trying but you can see why this is hopeless. In response to what he said:

1. I am not "off base." I know exactly what I am talking about. I used to be a senior engineer at a large ISP.

2. He did not carefully read what I wrote. I KNOW that the cable modems hand out 192.168.100.10 before they get connected to the cable network. I specifically discuss that in the text! That is not the problem that I have, and had he actually read what I wrote he wouldn't be confused about that. For the record, the problem is that after getting that IP from the cable modem, his network won't give me a real lease once the real cable network is connected.

3. That attitude from Comcast employees is exactly what makes this so frustrating. Their network must not be broken, therefore I must be "off base," therefore there is no reason to pay careful attention to what I'm saying. It is because they approach this with that sort of attitude that this problem will never be resolved.


 
RE: Comcast's DHCP is broken
by unmanaged at 5:16 pm EST, Dec 16, 2008

Decius wrote: I'd replace you with a DSL circit, but you and I both know that the phone company is just as bad, and they force you to buy a POTS line, which ought to be illegal.

Umm not true ... I just moved so I have not yet switched ISP/HSI providers... but AT&T does have DSL for 14.95 a month with no phone line...


  
RE: Comcast's DHCP is broken
by Decius at 5:51 pm EST, Dec 16, 2008

unmanaged wrote:

Decius wrote: I'd replace you with a DSL circit, but you and I both know that the phone company is just as bad, and they force you to buy a POTS line, which ought to be illegal.

Umm not true ... I just moved so I have not yet switched ISP/HSI providers... but AT&T does have DSL for 14.95 a month with no phone line...

hrm! I don't think that was available in my area when I checked, but thanks for letting me know. I'll check again.


 
RE: Comcast's DHCP is STILL broken
by Decius at 8:05 am EDT, May 12, 2009

Five months later and it happens again... my internet is down. The router (an SMC Barricade) that I hooked up during the previous debacle is now unable to obtain a DHCP lease from Comcast's network. This always seems to happen on a weekend...

So, I replaced the SMC with my Belkin. This is the exact same Belkin that Comcast's first and second level technical support insisted MUST be malfunctioning last December because it was unable to obtain a DHCP lease from their network at that time. It works just fine. I'm back up.

The SMC goes into my toolbox - inevitably in a few months I'll wake up and my Belkin will again be unable to obtain a lease from Comcast, and I'll replace it with the SMC, which will work just fine.

My previous explanation of this problem erred in terms of providing a lot of irrelevant technical detail about the difference between the DHCP process that I was seeing from my laptop and my Belkin. I'm now operating off the exact same Belkin that was not getting a lease in December, and it performs DHCP in exactly the same way that it did in December, but it now gets a lease just fine, and I now have a different device that cannot get a lease.

The simplest explanation is that Comcast's network is refusing a lease to my router's MAC address. Other MAC addresses can get a lease just fine. This condition appears to be temporary, but it lasts more than 24 hours and thats as long as I'm willing to wait for it to clear.

This is most likely a configuration problem with their DHCP server - my first guess is that it is not allowing devices with old leases to obtain new ones, but of course I can't troubleshoot this myself, and Comcast refuses to take it seriously. If I was a sadist I'd call Comcast's technical support so that they can tell me, again, that my router must be broken. I've got better things to do today.

I wonder how widespread this problem is. I wonder how many Comcast customers go through router after router thinking that they must be pieces of junk that fail constantly, when, in fact, there is a problem with Comcast's network. Hopefully, some victims of this problem will find this thread and be saved a lot of frustration.


  
RE: Comcast's DHCP is STILL broken
by Ken Arneson at 8:01 pm EDT, May 21, 2009

I just ran into the same thing today.

I have a Red Hat Linux box that suddenly stopped being able to get proper DHCP responses. So I unplugged it, and plugged in my wireless router instead. Same problem. Tried a different wireless router. Same problem.

So I call Comcast tech support, and just to humor them, I did all the stupid unplug/reboot stuff that I knew wasn't going to work. Same problem. Tried plugging my Mac laptop to the cable modem. Same problem.

OK, so I plug in and have them set up an older cable modem I have lying around. Linux box and wireless routers don't work, but my Mac laptop gets a web page that prompts me to install their software.

Against my better judgment, I install the software. Now my Mac laptop works, but only when it's directly plugged into the modem. But the Linux box, and the two wireless routers still can't get a proper DHCP response. So my laptop now only works if it's tethered to the cable modem. Not much of a laptop now, is it?

So I complain to them, and they suggest I get one of their new cable modems models (that I have to rent.)

Did that. Same thing--Mac laptop works, nothing else does.

I hope you're right in that the "blocking" of these MAC addresses is only temporary somehow. I'll try it connecting them again tomorrow and see what happens.


   
RE: Comcast's DHCP is STILL broken
by Decius at 9:57 am EDT, May 22, 2009

Ken Arneson wrote:
I just ran into the same thing today.

I have a Red Hat Linux box that suddenly stopped being able to get proper DHCP responses. So I unplugged it, and plugged in my wireless router instead. Same problem. Tried a different wireless router. Same problem.

Its weird that you had so many different devices fail to get an address. In my case, only a device which was recently connected would fail. Other devices which did not previously have an address could get one. However, it did require resetting the cable modem, which might have factored into your troubleshooting.

Some wireless routers have configurable mac addresses, so one strategy might be setting your router to use the mac address of your Macintosh.

Comcast's response with me was also "pay for our modem." I'm not going to do that until they can offer an articulate, technical reason why I should expect their modem to behave differently.


  
RE: Comcast's DHCP is STILL broken
by Decius at 4:31 pm EDT, Jun 2, 2009

Decius wrote:
Five months later and it happens again... my internet is down. The router (an SMC Barricade) that I hooked up during the previous debacle is now unable to obtain a DHCP lease from Comcast's network....

So, I replaced the SMC with my Belkin. This is the exact same Belkin that Comcast's first and second level technical support insisted MUST be malfunctioning last December because it was unable to obtain a DHCP lease from their network at that time. It works just fine. I'm back up.

And then three weeks later I'm down again. I was out of town on Sunday so its possible this occurred on the weekend as well. Tried to get a new lease with the Belkin and couldn't, even after resetting both the router and the cable modem. I replaced the Belkin with the SMC, and now I can get a lease just fine.

The last time I had to do this, I was up for 6 months. This time, I was only up for three weeks. Thats not a good sign...


   
RE: Comcast's DHCP is STILL broken
by flynn23 at 12:24 pm EDT, Jun 3, 2009

Decius wrote:

Decius wrote:
Five months later and it happens again... my internet is down. The router (an SMC Barricade) that I hooked up during the previous debacle is now unable to obtain a DHCP lease from Comcast's network....

So, I replaced the SMC with my Belkin. This is the exact same Belkin that Comcast's first and second level technical support insisted MUST be malfunctioning last December because it was unable to obtain a DHCP lease from their network at that time. It works just fine. I'm back up.

And then three weeks later I'm down again. I was out of town on Sunday so its possible this occurred on the weekend as well. Tried to get a new lease with the Belkin and couldn't, even after resetting both the router and the cable modem. I replaced the Belkin with the SMC, and now I can get a lease just fine.

The last time I had to do this, I was up for 6 months. This time, I was only up for three weeks. Thats not a good sign...

Comcast and most ISPs will tie a profile to your particular MAC address and the DOCSIS device used to terminate the line. This is how your line is provisioned and how your traffic patterns are monitored. These profiles are all automatically generated and the tech on the phone's ability to do anything with it is severely limited. If anything changes (ie power outage, upstream router reboot, etc) and there's any discrepancy in the profile tied to that MAC address, then you will see this behavior. It could be one of two things:

1) the MAC address of the device on your end has changed or was not the same as the one in the original profile. Connectivity can sometimes work once if this is the case, but not again if something burps the line. I see this whenever someone gets a new router or modem and doesn't call the ISP to have their profile updated or changed. This can sometime happen when an upstream policy has also changed, which will force a more "stringent" matching of device to provisioning profile thus fixing something that was previously working but considered "broken".

2) the profile itself is munged or the server hosting it has been borked. This is FAR more common, because these networks are constantly being upgraded, extended, and gear swapped. Policies change, and so do configs, which means that stuff gets hosed. And with 10's of millions of profiles network wide, something will inevitably fail. But good luck getting even Tier 3 support to a) check this and/or b) admit it.

On the DOCSIS side of the house, there's not a lot you can do here because of the way that profiles are stored and managed. On the DSL side of the house, you can sometimes jigger things with your modem to cause it to resync with whatever mechanism is managing the profiles (usually some form of RADIUS) depending on the modem itself. Older modems are ironically, more configurable. This is why simply changing the modem itself will usually solve things because it likely forces the provisioning infrastructure to "start over". However, changing back to your previous gear will usually cause more problems. It just depends on how that provisioning infrastructure is managed.

Another alternative I'd suggest is pay for multiple IP addresses for your line. Even temporarily. This will force a reconfig of your profile and likely will put you in a different queue for addresses, which might be enough to solve this problem permanently. Some providers offer static addressing, which would eliminate this problem altogether, but I don't think Komkast does.

One final thing to look at is the firmware of your modem. They do have bugs and they do get updated. Especially the DOCSIS junk. Updating it might help. It's a hassle but this is what you get for no competition in telecom.


    
RE: Comcast's DHCP is STILL broken
by skullaria at 12:48 pm EDT, Jun 3, 2009

I think it is their dchp.

I think it is possible they have some switches that can only handle so many mac addresses - so they can only give out so many IPs.

It can cause really funky problems when the address tables get full.
That's just what ***I** think. It is very rare but it happens, I've seen it happen. Depends on the switches, the older ones are more limited.

I had the same problem last month - I was down nearly all last month. I called repeatedly, I got bullshit answers. Then, every night - they go down around 2 am alot - I'd lose my connection AGAIN.

I would suspect my equipment too - and believe me - they were NO HELP with even trying to help me troubleshoot - insisting I was just a stupid customer that needed to recycle my modem...

I changed out ALL my equipment and basic cables except my modem - it wasn't any of that. I changed it back and got a refund on the new router "Sorry, I didn't need this after all."

I knew it was not network cards here, then mysteriously I was back up and fine again - and my DAD was down. Then HE had trouble on and off for 3 weeks, I know, because everytime HE lost his connection - he'd freak when an Outlook box would pop up and mistype his password.

My DAD is ok now - but last Sunday I get a call from MY SISTER - she switched to AT&T (she can!) because she [i]couldn't keep Comcast connected[/i]

No, something is up. It might not be what **I**think it might be, but something's up.
... once when I called I got a fairly intelligent sounding guy - he told me there were 12 people showing as being out on my loop. HE admitted they were having problems - but that it was NOT being shown as an outtage- this was all after the DNS problems that caused a lot of outtages stuff were all supposedly solved. My dad and sister are not in the same city.


    
RE: Comcast's DHCP is STILL broken
by Decius at 3:45 pm EDT, Jun 3, 2009

flynn23 wrote:
good luck getting even Tier 3 support to a) check this and/or b) admit it.

Thanks for this writeup - that is great information! I declined replacing the cable modem because it seems to function as an ethernet bridge once connected but perhaps there is something wrong with it. I'll get them to swap it out...


 
RE: Comcast's DHCP is broken
by Zortrium at 3:37 am EDT, Jun 20, 2009

Though I know you dealt with this awhile ago, after the last couple of days I feel your pain. I've burned through at least 15 solid hours trying to get my Linksys router to generate a DHCP request that Comcast's servers don't simply ignore. And the annoying thing is that any computer I plug directly into the cable modem has no problem getting a Comcast IP address (and I've tried several).

The most ironic twist is that the router (a WRT54G v.3.1) worked fine with the default Linksys firmware. But after a day or two (when it became clear how much of a piece of junk the default firmware is), I flashed it to OpenWRT, then later Tomato once I couldn't get OpenWRT to get an IP address. Under either, I've yet to be successful in getting the router to get a valid WAN IP address from Comcast.

I wasted a ton of hours researching the problem on the net, so I came across a whole pile of problems that people have apparently tried and succeeded with at various times. Though they've all failed for me, I'll compile them here for anyone else stumbling by.

1. All sorts of combinations of rebooting modems, routers, replugging cables, waiting minutes/hours while unplugged for things to reset, etc.
2. Clone the MAC address of the router to that of a host computer that CAN connect directly through the cable modem.
3. Minimizing the size of DHCP packets through the Tomato firmware -- some people thought that Comcast was dropping DHCP packets over a certain size. Again, doing this did nothing for me.
4. Messing with various other obscure router settings (firewall details, frame sizes, etc).
5. Enabling/disable STP on the router (spanning tree protocol). Apparently STP causes DHCP to fail on Comcast's network, but it was already disabled on Tomato. I tried enabling this then disabling again to no effect.

All I've been able to get is a constant "renewing DHCP lease" status on the firmware. The major thing that implicates Comcast's network is that my cable modem (some Ambit model supplied by Comcast) sees the DHCP requests from my router and forwards them, then logs errors when no response is ever received from Comcast's servers. So the next time I go look at my modem's logs, I see a whole list of "DHCP FAILED - Discover sent, no offer received" errors. So clearly, my router is properly communicating with the modem, and Comcast's servers simply choose to ignore the DHCP requests from my router for some reason.

At this point, I gave up and braved the trial by fire (Comcast tech support). The first guy I got on the phone went through the usual battery of "unplug/replug your stuff", then the usual "your modem looks fine to me, everything checks out", then got to what would become his mantra for the next half hour, "It's your router's fault. Change your firmware and/or your router. It's not our problem."

Of course, every time he fed me this argument, he couldn't give me a good reason why, if it was my rout... [ Read More (0.4k in body) ]


  
RE: Comcast's DHCP is broken
by Decius at 9:26 am EDT, Jun 22, 2009

Zortrium wrote:
In the meantime, my router is still as useless as it was 3 days ago. I'm sure I could just go buy another router and it'd work. But after sinking so much time into a project, accepting defeat from Comcast just sucks.

Thanks for taking the time to post this. It sounds like you ended up in exactly the same hole with Comcast's technical support that I ended up in. They simply don't understand that this sort of problem is possible - technically or culturally. Its good to know that I'm not the only one (although on some level I was sure that wasn't the case but I imagine most people don't know enough about their equipment to know that their router isn't broken and that Comcast is wrong.)

Two questions:
1. If you reflash with the linksys firmware can you get a lease?
2. Did you try changing your MAC address? And that didn't work?

I'm assuming the problem that I have is MAC based, as my way of dealing with it is to swap back and forth between two different routers everytime this occurs. Perhaps something more subtle is going on besides a MAC address change when I swap those routers, but without someone on Comcast's side to troubleshoot I'm kind of shooting in the dark.


 
 
Powered By Industrial Memetics