So there I was. Mouth open, cookie poised for entry, saying "Doh!".
I had used my own code for the router microcode. Of course it would be the same! My code was very efficient, and 'normal'. Doh!
What I needed was some microcode from a router that was from a system where I had noticed the slowdowns. That was the router microcode I needed, not my efficient (if I say so myself!) router code.
It was about 1:00pm, and I had to be at a client's business down in Sacramento at 2:30pm. And I realized that that was a client that had experienced some seemingly random slowdowns in their routers. I had some of their data packets that I had captured with my sniffer: I had used them in the router emulator program. I just needed to get the router microcode.
And I needed to finish moving the cookie to my still-open mouth. So I did, and the cookies were as good as ever. (Great bakery, Hilda's. Wonder if they are OK.)
I started getting ready to go to the client's office. While I was changing, I formulated a plan.
It wouldn't be too hard to dump the router's microcode to my laptop computer. I had full supervisor privileges on his system. He did have about 30 routers, though, so it would take a bit of time to gather each router's microcode. But it would not be a problem. I had to do some maintenance on their system at 5:30pm after the staff went home, and I had a few things to do before then. And there were always a couple of users that needed a bit of hand-holding.
So here's the plan. When I get to their server room, I'd write a quick program to grab the data from each router. Each one had it's own unique IP address, so I'd put those addresses in a table. A short routine to connect to each router, log into the router as the supervisor, and grab the microcode into a separate file on the server's hard drive. The microcode wouldn't take up very much space. I'd temporarily store it on the server's hard disks, then offload it to my laptop after hours. Not a problem.
I was ready to take off. I checked the computer room, and all the computers were functioning properly. I headed for the garage, setting the alarm system on the way out. Open the garage door (manually, remember?), and backed the LeBaron out of the garage, then shut the garage door and locked it.
I know, not a very fancy car to impress clients with, is it? It's a 1987 LeBaron, convertible. It was blue (my favorite color), and it was a convertible. It was in pretty good shape. I had gotten it restored a bit: overhauled the engine and transmission, new interior, nice tires (not too fancy), and an alarm system (electrical lockout, digital unlock via infrared port on my custom-made keychain). The car was in good shape, and it ran well. I didn't need a lot of room for equipment, just my battered old briefcase and the laptop, and a supply of floppy disks, along with the USB "Zip" drive for larger files. And I enjoyed driving the car with the top down.
I drove towards Interstate 80 (top down, of course), and the Elm Street on-ramp. Had to be a bit careful here, since it was a short on-ramp, and that part of the freeway was downhill, so you had to watch for semi trucks barreling down the freeway as you merged. Not much traffic today, so the drive down to Sacramento was fairly easy.
I drove down the freeway, through Newcastle, and down the hill into Sacramento. My client was near Cal Expo, the state's fairgrounds, over in a complex of office buildings right by I-80 and Highway 160. That part of I-80 is called the "Capital City Freeway", or Business 80, since the I-80 bypass was built on the north side of Sacramento. Headed through the infamous "Marconi Curve", which is usually jammed during rush hour. One of the local news-talk stations once did a promotion for their traffic-spotting airplanes, and you could get a bumper sticker with "I Survived the Marconi Curve" on it. It wasn't that bad, most of the commute's accidents happened on Highway 50, the road to Lake Tahoe on the east side of town.
I got to the client's office, and took a minute to talk to Sarah, the main receptionist. She was always good for a bit of gossip about how the computer system was working. I then poked my head into the boss' office, he had a few things for me to do. We talked for a bit, and I gave him my latest bill, which he quickly signed and asked me to take it over to Bob (the Accountant). Bob would have a check ready for me before he left for the day. Checks are good, and so is my service, so they didn't mind paying me right away.
I headed for the computer room. It wasn't really fancy, nothing like 'Mission Control'. It was just a small office (locked, of course, and it's own air conditioning system) with the servers, and the racks of routers, and the hubs with their nest of cables. They were a billing service, and were connected to the Internet via a T-3 line so they could get the billing data from their clients. I had set up the data collection routines from their clients (some of whom were also my clients), so I had good access to all their systems. I'd sometimes telnet (connect by modem) into one of their systems from home to perform some maintenance. That was a fairly common task for me, so I had full "root" access.
When I set up the connections to the other businesses that the company processes customer bills for, I set up a simple little logging program that would report on the connections to their various clients. The log program would help alert them to missing connections, etc. It also reports on slower than average connections, because it keeps track of all the connection times over the last six months. The boss had noticed that the log printouts had shown that some connections were a bit slower than usual. I told him that there could be many reasons for that. The reasons I told him were fairly benign: congestion on the Net, more data than normal, stuff like that. Nothing to worry about, I told him, but I'll check out the equipment and the lines to make sure all is working OK. That's what they pay me for.
I logged onto the system terminal with my usual user ID (not one of the stealth users that I had set up; I only use those when I need "special" access), then su'd into a 'root' account. The root account gives me full privileges into the system. I can work with user accounts, start programs, compile source code, etc. I then took a look at the log files, which also track the connection or router used. By keeping track of the router used for the various connections, I could also track problems with the router. After a few minutes of analyzing the log files, I noticed that one particular bank of routers was showing more slowness than the other ones.
Most of the routers we had at that place were from the same manufacturer. The problem ones had been recently purchased. They were newer models, supposedly faster and more efficient in their routing activities, according to the specs. But they were occasionally showing common slowness in data transfers. Not much that you would notice, unless you kept track of the transfer times like my logging program did.
The company wasn't due for a big file transfer until later that night. So I disconnected one of the suspect routers from the network, and hooked up my laptop computer to one of the router's diagnostic ports. Since I was suspecting some nefarious activities ('nefarious': that's another good word) on those routers, I immediately dumped the router BIOS into a file onto the laptop. I also got the routers' log file.
There were 10 routers in the new batch that the company had purchased. Well, OK, I told them the brands to get, but they had a good reputation for their products. I connected my portable to the other nine routers, and dumped a copy of their BIOS and log files into separate files on my laptop. Then I reconnected them to the system, and activated them again so they would be ready for that night's data transfer.
I hooked up my laptop back to the network, logged in again, and wrote a short program to catch packets that go through those routers. I set up a hidden directory in my personal (well-protected) area of the server's hard disk volume, and set the program so that it would copy those packets into those directories. Since there was a lot of data that would be transferred that night (and every night), I also set up the program to compress the files once every 20 minutes. And I limited the total size of the compressed files so that I wouldn't fill up the server hard disks by accident. Luckily, the company had lots of disk space available; I had made sure that there would be enough disk space for future growth of the company.
I compiled the program, and had it do a test run with some router data. It worked just fine, so I did a quick double-check of the size limiting routine and the compression routine, then set the program to run during the night's data transfer. I also modified the logging program so it would keep a separate log file that only I could access. That log file contained additional information about data packets captured, such as the data packet's original data source. A quick test of the shadow log file program, and then I set it to be active all the time.
I also double-checked my back doors into the system. Notice that 'back doors' is plural. I had set up multiple back doors just in case one was discovered. Not very likely on this particular system, since I did most of the maintenance, but it was a good precaution. I usually also set up some more visible back doors, so that they would be the first to be found if somebody else happened to be looking. (At one client that suspected hacker attacks, I "found" one of my throwaway accounts as proof of an attack, along with a couple of real accounts from another hacker that tried to get past the firewall that another consultant had set up. They were impressed with my skill, and I got another client. Their system was much more secure when I got done. Although my backdoor accounts were also installed, just in case they were needed.)
I had also modified the logging subsystem of the operating system to ignore any system activity when I run certain commands. These commands are not found in the OS, so they usually give an error. Sort of like typing in a command of "lss", when the correct command is 'ls' to display a list of files in a directory. If you typed in a command like 'lss', the operating system would come back with a 'no such program' type of error.
On my systems, a command like 'lss' is the key to the first sequence of stealth commands (of course, that's not the command I used). When I typed in a specific series of unknown commands, that series of commands is recognized internally as the command to start the stealth part of the OS that I had modified. With the 'stealth mode' on, any further commands would not be logged as they were executed. Another series of unrecognizable commands would turn stealth mode back off, so things would be normal. There was even a "stealth off" timeout so that things would go back to normal if I was interrupted as I was doing some back door work while accessing the system from a remote location.
So I entered the "stealth on" mode, and made sure that all my back door accounts were still valid. If any were missing, I'd be a bit worried. They were all OK, so I cleaned up my tracks of the shadow programs, copied them (after encrypting them) to a floppy for safekeeping (the label on the floppy was innocuous to the casual viewer), then turned off the stealth mode. I typed in a few more random commands that made it look like I was looking at log files, etc., like I normally do when maintaining a system. A quick double-check of the system to make sure all was OK, and I logged off the server. All the routers, including the new set of 10 that were now being monitored, were hooked up and ready to go. So I was done here.
I was ready to leave, when I decided that I wanted to take one of the new routers back home for more extensive testing. I logged back into the system, did some rebalancing of the routers so that all normal processing files from the company's clients would be transferred through the remaining routers, then logged off. I took one of the routers out of the cabinet.
I stopped by the boss' office, and told him that the routers seemed OK, that all diagnostics passed, although one router looked like it might be failing. I told him that I was going to take the suspect router back to my office (well, really my home) and run some more extensive long-term diagnostics on the router. I explained that the missing router would not cause a problem, because we had designed the system for some redundancy, and I had rebalanced the system load to take into account the missing router. He was pleased, of course, with my excellent service. I told him that I'd be back in a week to bring the router back, and to do another scheduled maintenance on his system. That was OK with him: as long as the data was being transferred and he could process the data (and bill his clients), he was happy.
I made a quick stop by Bob's (the accountant) desk to pick up my check, and I was off after saying goodbye to Sarah (the receptionist). I got back into my car, put the top down, and headed back up the hill to my house. By that time, it was a bit after 6:00 pm, so I made a side trip through the McDonald's drive-through for a couple of cheeseburgers and a large Hi-C orange drink, and back up I-80 to Auburn. By this time, there wasn't too much traffic. Not like on a Friday night, when everyone heads for the hills for the weekend. A trip from around Cal Expo to Auburn on a Friday night can take an extra 45 minutes to an hour to go 30 miles.
It had started cooling off by the time I hit Roseville. The delta breeze was kicking in, bringing that cool air from the ocean through SF and Fairfield. That's what we call the 'valley air conditioner', so even on hot days, it usually cools down at night to a comfortable 55-65 degrees.
I stopped for gas at the Arco at Newcastle. Their prices were a bit higher than down in the valley, but they did have 49¢ frosties. Chocolate and vanilla swirl, of course.