Thirty-four years in IT - The System Office, Novell Directories, and Building a State Backbone (Part 3)

Unfortunately nearly all the work we put into administrative and academic technology had to be abandoned. As a part of a larger initiative across the state, the various colleges and universities were being merged together into a single system that today is know as Minnesota State. In that process our college president retired, and the new college leadership de-emphasized the use of technology In business practices. Additionally, I recognized that at merger time most of the software that I had written would not usable. So I spent some time getting us off the software I wrote and on to other software that I knew would be used post merger.

In a lot of ways that was a set back for both the college and the students. It was many years before faculty and students would have the functionality we had in 1993.

In the summer of 1995, as the colleges were being merged into what is now Minnesota State, I decided that since the college was not emphasizing technology as much as before and since I was ready to leave the small town environment, I would move to the system office and see what opportunities would be available there.

I became a Novell System Administrator and built out an NDS tree that turned out to be far more similar to modern directories then what It was to the best practices of the time. Conventional wisdom was that you divide up your directory by business unit or business function and create the user, printer file share, and other and directory objects related to the business unit in the OU for the unit. I quickly realized that conventional wisdom and the Novell Directory design guides were wrong. The 'organization hierarchy'  style directory architecture didn't make a whole lot of sense, so I implemented a relatively flat directory where the only OU's were at a very high level and only used to categorize types of objects rather than business units, departments or divisions. The flat directory Is something much closer to what you'd see today and something like Azure Active Directory.

That directory had continuous up-time - I.E. you could log into the directory -  for over a decade. Of course the servers providing services we not always up, but the directory was. :)

Conventional wisdom isn't always right.

While troubleshooting a statewide IPX routing problem I realized that In the post merger chaos, the wide area network was essentially unmanaged. There were routers all across the state on either 56 kbit circuits or T1's running along without any active oversight. So I grab the Cisco manuals, taught myself IP routing, asked a co-worker for the passwords, and started managing the wide area network.

That meant physically locating all the routers around the state, documenting circuit locations and circuit ID's, cleaning up routing tables, cleaning up configurations, and getting the hardware on common operating system versions. We also created a common campus network edge design, wrote our own database driven network monitoring package, combined monitoring and CMDB databases, and built out a system of thresholds, triggers and alerts to help us keep tabs on the hundreds of devices and dozens of sites. Written in Perl and leveraging MRTG, of course.

To help us monitor the network we push purchased Netscout probes for every site. Because we had more than 50 probes, and GUI's don't scale well, I dug into the Netscout management software and figured out that the graphical user interface was just a skin over the top of some very powerful command line programs.  I wrote a package around those programs that automated the maintenance and monitoring of probes and automatically gathered 'Top N' IP, TCP host and port counters for their network traffic data from the probes. I exposed that data to campuses so that campuses had a pretty complete view into their network traffic readily available to them. My idea was that if they knew more about what was happening on their network, they'd be able to do a better job of running their network, and most importantly - they'd be able to resolve more of their own problems themselves. And call me less often.

As the wide-area network evolved and became more critical to our operation, we had to make significant investments in bandwidth and availability. Making those investments on our own without a partner turned out to be very difficult, so we partnered with the State of Minnesota to leverage our resources and their skills to build a common statewide backbone usable by all State agencies and our system - Minnesota State. For nearly twenty years, the State of Minnesota has been the backbone provider for Minnesota State Colleges and Universities. As the State had already partnered with the University of Minnesota, the three largest public entities in the state all  share a common state backbone and Internet connection. In the partnership, Minnesota State benefits from the resources and expertise at the University of Minnesota and State of Minnesota, allowing us to concentrate our skills on other aspects of running a state-wide enterprise.