Nevis Network Change

This is a discussion of the issues we've identified in the planned Nevis network changeover on Wed 08-Oct-2003. It is a work in progress. Please send me your comments and questions. I'll try to keep this document up-to-date with what we've decided.

Overview

On Wednesday, October 8th, we are planning on migrating our existing T1 network to the new T3 service. The switch over will require that every system in the lab will get a new IP address.

Columbia has assigned us the range 129.236.252.0/24 and we will simply replace the 192.12.82.xxx with 129.236.252.xxx so the last octet will remain the same. There is one exception: the network gateway changes from 192.12.82.5 to 129.236.252.1.

On Wednesday morning, at approximately 9:00am, we will pull the plug on the existing T1 network and start the migration. We expect that the bulk of the work in switching over the IP addresses will be completed in one day and SOME network services will become available midday. There will be limitations as the Domain Name Servers as they take approximately 24 hours to repopulate the router databases around the world. If you know the IP address of a node you need to get to, you can us the IP address rather then the URL provided you have network services.

Mail will automatically spool up on the backup mail server located downtown at the Annex. Once our mail server is back on line, mail will automatically be synchronized. There will not be any lost email during this process.

Devices that obtain their IP address via DHCP such as laptops need only to reconnect to the network and they will receive the new IP address.

If you maintain your own systems, the IP address of the printers will need to be changed as well as any SSH sessions or Xterm sessions. Again, simply replace the 192.12.82 part of the address with 129.236.252 or if you are referencing the address by URL, simply wait until the URL is repopulated.

Bill and Dave will be visiting each system in the lab to ensure that they are configured correctly. If you need assistance with your system, please make sure to see us either late in the day on Tuesday or Wednesday morning.

Priorities

The general tasks we have to consider are described in the following sections, listed in no particular order. The actual order in which we'll perform these tasks is roughly:

  1. Network tests
  2. Firewall configuration
  3. Main network switches (network room, Room 119)
  4. Admin systems (franklin, hypatia)
  5. Domain name services (franklin, hypatia, annex)
  6. Mail services (franklin)
  7. DHCP services (ntserver1)
  8. Multi-user systems (nevis1, sasha, kolya...)
  9. Desktop systems (anna, marie, tanya...)
  10. Systems in the electronics area and machine shop (atlase1, pellinore...)
  11. Modem pool
  12. Laptops
  13. Routers (including the wireless network)
  14. Printers and print services
  15. Web services
  16. X-terminals

With any luck, most of these tasks will be completed on the day we switch. The last third of the list probably won't be finished until the following day.

UNIX cluster

General systems

The IP address of a Linux box is set via these files:

/etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network /etc/hosts

It's easy to come up with a script that will automatically edit these files (something like:

perl -pi.bak -e "s/192\.12\.82/129.236.252/g" `find /etc -type f -exec grep -rl 192.12.82 {} \;` ...where 129.236.252.0/24 is our new class C network). After the script is run, the system must be immediately rebooted. Probably we can just copy this script to the /tmp directory on these systems and run it from there.

Note: Special handling is required for the GATEWAY value, which is changing from 192.12.82.5 to 129.236.252.1. The value can be set in either /etc/sysconfig/network and/or /etc/sysconfig/network-scripts/ifcfg-eth0.

Other typical files that contain 192.12.82:

/etc/hosts.allow /etc/resolv.conf /etc/printcap

In the case of the first two files, I'll have to prepare "transition" versions that sllow for both 192.12.82 and 129.236.252 networks to be valid. We can remove the 192.12.82 portion after the transition.

Special servers

The NIS master server (hypatia), the mail server and primary DNS server (franklin), and the web server (ada) contain many files with the 192.12.82 specified. It's not hard to edit these files automatically with a similar script to the one above, but once this is done the services won't be available on the old network.

In the case of the mail server, the Annex server will store the mail while it's down. I don't anticipate that we'll lose any mail.

Here are some of the files that would have to be examined on nevis1:

/etc/config/static-route.options /etc/init.d/network.local /etc/bootptab /etc/sendmail.fc /etc/ethers /etc/hosts /etc/ppp.conf /etc/nsswitch.conf /etc/cups/printers.conf /etc/snmpd.authfail /etc/resolv.conf

Annex systems

The following files will have to be transitioned on Annex systems as well: /etc/hosts.allow /etc/resolv.conf

However, I don't think the Annex systems will even need to be rebooted.

The Annex server refers to the Nevis network, since it acts as a print server for the Nevis printers, acts as NIS slave server, etc. Again, I don't any of these changes will even warrant a reboot.

Windows systems

(Need input from Dave here.)

Firewall configuration

The firewall must be configured for the new network. Hopefully, it can be as simple as unloading the current configuration to a file, globally changing 192.12.82 to 129.236.252, and loading the modified file back onto the firewall.

Due to CU computer administration policy, the gateway address on for all systems must be changed. It was 192.12.82.5; it becomes 129.236.252.1. Note that this is an exception to the general rule that the last octet of the IP change will not change due to this transition.

Other hardware

We'll have to go to each of the following and manually change the IP addresses:

Domain Name Services

I think this may be the part of the network change that will be the most difficult to manage.

The current situation: Columbia Computing has delegated the nevis.columbia.edu sub-domain of columbia.edu to us. They've made all the arrangements so that our current nameservers (192.12.82.8, 192.12.82.7 = franklin, hypatia) are the authoritative servers for this domain. They've also designated secondary servers to "mirror" our information:

# 128.59.59.218 = saell.cc.columbia.edu # 128.59.59.70 = kedu.cc.columbia.edu # 128.59.39.39 = diaduit.cc.columbia.edu # 141.211.125.15 is tickleme.mr.itd.umich.edu, a secondary server at U Mich

However, they've done all this for the 192.12.82 network, a set of IP addresses that they do not manage.

For the network change, Dave convinced them that we should be allowed to continue to manage a nevis.columbia.edu domain for a set of IP address that they do manage, which is not their normal policy -- but makes life much easier for us.

Some issues:

DHCP Services

Right now the NT Server provides the DHCP services for both the "upstairs" groups and Astro/RARAF. Sharing the limited pool of dynamic addresses has always caused us problems, and we're looking forward to placing Astro/RARAF in their own DHCP address pool (in the long term).

Some issues:

Astrophysics/RARAF

We will co-ordinate with the Astrophysics and RARAF system administrators; the Astro/RARAF systems will switch networks along with us.

Presently we manage their DNS names, since they're in the nevis.columbia.edu domain. For the current network transition, we're going to "take them along with us" and leave their connection to our network services unchanged (DHCP and firewall).

Long-term issues: