Memorandum 13-130 IT Infrastructure Project

Memorandum 13-130
TO: City Council
THROUGH: Walt Wrede  - City Manager
FROM: Nick Poolos – IT Director
DATE: 9/4/13
SUBJECT: IT Infrastructure Project

Phase 1:  Server Virtualization and Consolidation

Server Virtualization is using a special software system called a hypervisor to run multiple operating system instances on a single shared physical server.  The current performance of an entry level server is more than is needed for any single application.  This leads to a rack of mostly idle servers that are still drawing power 24 hours a day, 365 days a year.   Running multiple servers is a requirement due to compliance rules and security concerns.  For most city systems there is a need to separate certain datasets and users.  Also running applications on separate servers isolated vendors from each other and prevents one vendor from blaming another for all issues.
Virtualization also enables some IT efficiencies as the "server" is now isolated from the specific server hardware and is running on a uniform and generalized platform.  This allows the virtualization software to pick up and move a running server from one physical host to another and also mirror a running system to a remote site for disaster recovery.  The moving of a running server requires that both physical servers be attached to a shared RAID storage system.  The ability to move running servers allows IT to maximize the utilization of the servers and minimize downtime.  These same tools will allow for point in time copies or snapshots of a running server.  Snapshots can be made before any major change, patch or upgrade allowing for a quick recovery in case the changes don't go as planned.   This is very important as the City does not have the staff or servers to maintain test environments for our applications where such changes could be tried before attempting with the production server and data.  Snapshots are also the key to quick and constant backups.  Currently the full backup of the city servers takes about 50 hours this is a concern as the city amasses more data this window will continue to expand.  A snapshot can complete in seconds and then the backup software can copy the snapshot to the backup media.  This allows for a consistent view of the server and data and since nothing is changing in snapshot the copy can be made quicker.  
As a server now really just looks like a program running on the hypervisor,  IT can make use of configuration templates.   This will cut a new server installation time from 6 to 8 hours to 10-15 minutes.  Also vendors are starting to ship "virtual appliances" that are preconfigured servers that install into a customer's virtual infrastructure.  The city has already acquired one such system and is considering several others.
Currently there 4 new servers planned.  If virtualization does not occur, the average cost of a server is $2200 --with virtualization the additional servers are covered.

Server Virtualization Project Costs
Shared RAID Storage System $35,000
VMware Virtualization Platform $15,000
Virtualization Enabled Backup $15,000
CPU and Memory Upgrade $2,500
Total $67,500

Phase 2:  Microsoft  Server and Client Management Tools

Currently the City has McAfee antivirus licensed.  This software costs approximately $3100 a year in maintenance.  Over the past year McAfee has missed several pieces of malware costing the city approximately 70 man hours both in IT time and lost productivity while the workstation was unavailable.  The missed malware is bad enough but McAfee has been actively blocking software updates, including Flash, Acrobat and Java, and installs.  IT has spent at least 150 man hours working through these issues.  The blocking of Flash, Acrobat and Java has been extremely concerning as there have been numerous wide spread attacks on these applications in the past year.
The client management tools are in a Product called “System Center Client Management Suite”.  These tools will allow IT to automate client update processes, improve client PC security by restricting the normal user access levels, an ensure compliance through unified security software, patch management, and reporting.

Server and Client Security and Automation
Microsoft Enterprise CAL (110 FTE)  $15,000
Microsoft Academic Desktop w/ Enterprise CAL (Library) $3,000
MS Server components and Professional Services $12,000
Total $30,000
This solution needs several new servers to run and thus requires phase 1.

  Phase 3:  Wireless Metropolitan Area Network (WMAN) Upgrades

Currently the City has 7 sites connected with 802.11a/n (Wi-Fi) radios.  Data rates vary by site but are in the 12 – 40 Mbit/s range with 12- 15 Mbit/s being typical of most sites. These radios are using  the “unlicensed”  5 Ghz band.   There are a very limited number of channels available and they all have power restrictions (800mW, 250mW,200mW).  By FCC rules all users of these frequencies must coordinate and resolve conflicts.  When the city started using these radios 5GHz was not often found in consumer devices.  This situation has changed as many devices have added 5GHz radios and more 802.11 devices are appearing everyday.
There are FCC licensed radios available where the end user obtains an exclusive license  for a given frequency on a given link.   The FCC maintains a database of these connections and acts as the frequency coordinator, thus guaranteeing an interference free link.
The other issue the city is facing are mandates that networks outside of certain secured buildings are encrypted by devices that have been validated to the FIPS 140-2 standard.  Currently the city WMAN links are encrypted but the devices used have not been validated to the FIPS 140-2 standard.  The first  area to fall under these compliance mandates is Homer Police Department.  
IT proposes building out a FIPS 140-2 validated WMAN  making use of exclusively licensed FCC frequencies.   The primary links should be in 350Mbit/s to 1 Gbit/s range.  This will allow for further server consolidation of department servers into the virtual infrastructure.  The exact radios and frequencies used will need to be developed during the licensing process. The primary network will be backed up by a citywide 4.9 GHz public safety network for security cameras, communications, etc.
The expected service life of the radios is 6-10 years depending on the model and upgradability.  The FCC licenses have 10 year duration and will need be renewed at a reduced cost at that time.

Wireless Metropolitan Area Network
Microwave Point-to-Point Radios (FIPS140-2)  $100,000
FCC Licenses (7 or 8 Links) $35,000
4.9 Point to Multi Point Base Stations $18,000
4.9 “Subscriber” Endpoints $7,000
Mounting and Installation $20,000
Total $180,000