Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I spend a decent amount of time investigating trouble reports, and in my experience it's quite uncommon to even get as much information as was provided in what google showed. It's also fairly uncommon to get any of these sorts of rare configurations in trouble reports, and usually takes some probing.

When I was a new engineer working in telco, one of the longest investigations I worked was when connectivity broke between one of our regional roaming partners and 1/3 of our nodes (I'm summarizing to try and keep the story brief). We called them and asked if they changed anything, reviewed the configuration and secrets used on the tunnels, etc. And were working with the vendor to go through any problems with the implementation. Saturday morning and probably 20 hours of investigation later, a new engineer at the regional partner see's there is a work order for changes to the connectivity to our nodes (we were adding some new ones) that was supposed to be executed that week. A typo in the change overwrote the secrets used by an existing tunnel instead of creating a new secret for the new peer. The person we were working with to investigate, was the person who implemented that change and told us several times nothing changed. He was also the one we worked with and read through all the secrets for typos or issues and didn't notice anything. Saturday morning he get's into the office, is shown the work order, and goes, oh yea, I did that at exactly the time the tunnel went down. Fix of typo'd secret later and everything comes right back up.

So just in my experience, I find it quite plausible that buffer size was not mentioned. And even besides this story, I know I've personally missed connecting causes with potential effects when investigating a problem, it's very easy to dismiss some setting, like the buffer size, as being connected specifically to DNS behaviours, especially if they are not noticed together or with a strong change management system that helps connect the timelines together.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: