The High Cost of Computer Worms
Wednesday, March 17, at 1:30 p.m., U.S. Eastern timeColleges are spending increasing amounts of time and money to fight computer worms and viruses. What can security experts do to combat the spread of infections on college networks?
A recent survey of 19 research universities shows that each institution spent an average of $299,579 during a five-week period last summer to fight off the so-called Blaster worm, one of many infections that have crippled college networks.
College computer administrators say they are spending too much time and money trying to find and destroy worms and viruses. Colleges are hiring more security experts, scrutinizing their networks for problems, and diverting resources away from information-technology improvements to deal with computer infestations.
What are the most effective practices colleges can use to prevent their networks from becoming infected? And how should they respond once an infection strikes?
» Colleges Brace for the Next Worm (3/19/2004)
Gregory A. Jackson, vice president and chief information officer of the University of Chicago, has been active in several college information-technology organizations, including the Common Solutions Group, a coalition of chief information officers at 25 universities. Mr. Jackson surveyed some of the group's members to estimate the cost to universities of fighting computer infections. He has also been a faculty member at Harvard and Stanford Universities, and at the Massachusetts Institute of Technology, where he served as director of academic computing.
Mr. Jackson will respond to questions on Wednesday, March 17, at 1:30 p.m., U.S. Eastern time. Questions and comments are welcome and may be posted now.
Andrea L. Foster (Moderator):
Hello, I'm Andrea Foster, a staff reporter here at The Chronicle of Higher Education.
Today we're going to be discussing what colleges can and should do to combat computer worms and viruses. Our guest today is Greg Jackson, chief information officer at the University of Chicago.
Welcome, Greg.
Question from Patrick Whitaker, College of William & Mary:
In discussions with a colleague, we anticipated the possibility that the need for protection from worms and viruses would eventually shut down the internet as we know it. We would be left with individual private networks and limited access to those outside. Do you see this happening and if so, how soon? Thank you!
Gregory A. Jackson:
I don't see this happening, no. Certainly it's harder to manage networks and machines in the presence of worms and viruses, and there's an endless arms race to stay ahead of the bad guys, but in the long run this increases the cost of the whole shebang rather than renders it inoperative.
Question from Barbara Belle, University of Dayton:
1) Do other Universities ever block specific file types (i.e.,files with .scr or .pif extensions)?
2) Do other Universities have minimum standards for the machines they grant access to their networks? Are they doing anything to test those machines (virus protection and/or Operating System updates)?
3) Do other universities have a "Security officer" position, and who does that person report to? If not, Who/What person in what role at most universities makes these types of decisions?
Gregory A. Jackson:
1. Yes, most colleges and universities block things. It varies from place to place -- most, for example, block executables (.exe, .pif, screensavers, etc), some block certain specific files, some block lots of stuff. And the effects vary, in terms of bad stuff caught and good stuff junked.
2. Most places have recommendations, but few have actual standards, and fewer still have formal licensing, review, or audit procedures. Then again, most of us to a certain amount of scanning for specific vulnerabilities, and we often have very particular requirements we impose on machines we find to be vulnerable or compromised.
3. Security officers report lots of different places and lots of different levels. Mine, for example, reports through my Networking organization; Indiana's reports directly to the CIO; other institutions choose Internal Audit or other admninistrative lines as the right place. It's not a well-defined job or organizational position.
Question from George Brophy, University of Hartford:
Is there a "rule of thumb" on technical resource allocation per number of students to support antivirus, antispam, and network security functions?
Gregory A. Jackson:
Not really. It depends in part on institutional strategy. Some places put all their resource into network configuration, for example isolating dormitory networks; others, such as us, focus on host-based strategies such as site licenses for antivirus and firewall software, plus lots of user education and security scanning for problems. Here we have about 6 network security folks, which at about $100k per with benefits comes to $600k, and we probably spend another $300k on network-security devices, scanners, software licenses, and so forth. So say a million bucks a year all told, out of a total University budget of $1-billion and IT expenditures of about $65-million (including computer purchases and such). My guess is that most places aren't all that different, although the expense may be divided up in different ways (by School, for example) and thus harder to identify and aggregate.
Question from Lisa L. Spangenberg, UCLA:
Given that there are no viruses or Trojan horses for the current Macintosh system, OS X 10.3, and given that it is essentially UNIX, and given that the most common applications (Microsoft Office Suite, Adobe applications) work very well on OS X, why don't more institutions adopt Macs and encourage faculty to use them?
Gregory A. Jackson:
Well, first of all, there are viruses and Trojans that afflict MacOS, witness Apple's periodic release of security fixes to counteract them. But the small installed base of Macs makes them an unexciting, low-visibility target for the bad guys, and so the weaknesses don't get exploited much. In the case of Unix, the vulnerabilities are greater -- even in the Mach kernel underlying MacOS -- but once again the installed base makes for an uninteresting target. If, as you suggest, suddenly Macs were much more widely used, they'd rapidly become an interesting target, and we'd see more bad-guy action. An interesting consequence of this would be a focus on Apple's policy for security updates, which is approximately that after a brief while you have to pay for them. But I digress.
As to why we don't recommend more Macs anyway, which isn't really what you were asking but what the hey, there are two vexing and continuing problems: it's becoming harder and harder (and hence more and more expensive) to find qualified Mac technicians and support staff, and Macs themselves, with a couple of exceptions (such as iMacs and low-end iBooks), remain stubbornly more expensive than their Windows or Linux competitors.
Question from Kevin Shalla, U of Illinois:
Given that users cannot help themselves from being persuaded to do things they should not (like open attacking attachments), and that antivirus vendors can only identify these attachments after they have already attacked many computers, can we return to a day when email is less functional, but totally safe by excluding attachments?
Gregory A. Jackson:
And a world without SUVs, cell phones, and handguns too, right? The answer is, no, we can't go back. Rich email -- the kind that can carry all kinds of stuff like documents and programs and pictures and music and URLs and you-name-it -- is something people value immensely. Truth is, we've let people think it's easier and cheaper than it actually is. But the solution to that is going to be increased costs, not decreased functionality. It's not that hard, with a modicum of effort and expense, to render one's email quite safe without crippling its capabilities. That's the right answer, not going back to a simpler time.
Question from Ricky Streight, Ivy Tech - Wabash Valley:
With this type of fiscal impact, why haven’t colleges and universities adopted an approach similar to software quality assurance in that an ounce of prevention is worth a pound of cure.
My experiences tell me that you’ll spend the money one way or another and you’ll spend much more on the back side. What types of researched “security quality assurance” methods or principles, if any, have been identified (beyond the obvious like firewalls) that enhance data and system integrity at all points in the project life cycle?
Gregory A. Jackson:
My President is fond of saying that he's the CEO of a $2-billion corporation yet can give no one a direct order and expect compliance. He's exaggerating, of course, but his point is valid: Universities and colleges are oddly decentralized places, with concomitantly Balkanized policies and support activities. We'd clearly be better off taking some of the decentralized remediation expense and investing it in prevention, but those who now have that decentralized activity don't want to give it up. And Central doesn't have any spare cash lying around, so we can't go ahead and spend more on prevention. It's a nasty chicken-egg problem.
The simple answer, in other words, is that we don't have the organization necessary to redirect resources as you suggest, even though that's the right thing to do. Of course, this isn't only an IT problem...
Question from John Gerone, Tulane University:
Based on your experiences with the CSG what can the IT community do beyond installing and configuring the following listed items below. NOTE: We find laptops users and students to be a challenging group of users to establish IT policy for. 1) Automating critical updates 2) Virus walls 3) patch pushing (shavlik) 4) Router blocks for, ICMP activity
Gregory A. Jackson:
Well, to that I'd add thoughtful and careful scanning for vulnerabilities on network-connected hosts, plus widely distributed and centrally-managed intrusion detection to find compromised machines and detect attacks quickly enough to slow or stop them. Those last quickly raise privacy issues, and those deserve careful discussion. But those who manage networks need to have not only the user tools you listed, all of which I agree are important, but also more general tools to monitor and manage what happens on the network.
Question from George Brophy, University of Hartford:
Have universities reported any particular successful student antivirus programs (procedures as well as software)? What student penalties do some universities enforce if policies or procedures are not followed?
Gregory A. Jackson:
At this point I'd say lots of universities share our experience here, which is that site-licensing antivirus software, strongly encouraging people to leave it in frequent auto-update mode (at least daily), and rebuking -- strongly rebuking -- those who cause problems by leaving themselves vulnerable is immensely effective. Except for a few instances when viruses spread rapidly before the DATs caught up, we've had very few instances of individuals firing viruses and causing network problems here (he says, going against his heritage and knocking wood).
Question from Michael Hirsch, New Politics, New School GF:
People talk about worms and viruses as if they were organic compounds, diseases to be treated medicinally. But even in medicine, such aspects as environment, nutrition, the availability of public health services and other socially influenced factors are part of the solution to curbing epidemics.
My questions is: where are these viruses and spybots coming from???? Somebody is making the viruses and worms. For what purpose? In whose interest? And if it's just vandalism, then why weren't there similarly wide-ranging attacks in the past, attacks directed at other technological breakthroughs. (Outside of strikes and Indian wars, I don't know of any systematic disruption of railroad service in the 19th C. No bombings of the interstate hwy system in the 20th, or jamming of radio or TV sugnals, etc.)
So why is there a vandal class out there, if there is one when there never was one before. (And no fair citing the English Luddites; they only attacked large looms that put small weavers out of work.) Or are computers just so much more vulnerable (if indeed they are)to individual pranksters? Not that I doubt it's a problem, mind. I just installed SpyWare, on top of my Norton 2004. What a pain!!!!
Gregory A. Jackson:
Well, the heart of the problem is that the architecture for the Internet was designed to solve a different problem than identifying users, and therefore it makes anonymity very simple. Also, in contrast with the highway and railroad analogies, someone who vandalizes the Internet becomes internationally visible, whereas someone who vandalizes a highway -- consider the Columbus sniper, for example -- has to go pretty far before he (it is a he, arrested yesterday) really affects that many people directly. That is, the Internet permits very broad impact while providing anonymity, and that is a very dangerous combination.
As to where they're coming from: lots of theories, not much data. Vandalism is one theory, revenge another, rehearsal for terrorism is another, extortion is another ("look what we did to the University of Chicago, if you don't want to be next, here's the Swiss bank account number to pay"), bored script kiddies is another. The current viruses are awfully well done, both technically and in terms of social engineering, so most of us have come to discount script kiddies, but we don't know much more than that.
Question from John Lawson, Tulane University:
Greg, given that much of this issue (at least for us) occurs with residential student computers, has your institution considered physically separate networks for residential students? If so, would you outsource the management of that connectivity?
Gregory A. Jackson:
We've considered it, but frankly our location doesn't provide a lot of options. Moreover, it doesn't seem to me that simply turning this problem into one where students are beating up on one another is the right answer. The Internet is central to education today, whether we like it or not, and I don't think we can duck our responsibility to students by locking them all in a (network) cage together and letting them work things out. I much prefer solutions that involve active management of networks, including packet shaping and other such strategies. As to outsourcing: I may be living a dream, but I don't know anyone who knows how to manage open networks as well as the typical college or university. Lots of entities know how to manage closed networks with strong authority, hierarchy, and enforceable rules, but no college or university approaches that model.
Question from Diane Sullivan, U of Northern Iowa:
For our on-campus students and staff/faculty, we provide enterprise virus protection. For off-campus, home users, do you see a cost/benefit to using open source software, including Redhat Linux, Open Office, etc. that are less targeted?
Gregory A. Jackson:
Simple answer: No. Many of them are much harder to support, and when they get sick, they're harder to diagnose and fix. Plus doing this would simply increase the heterogeneity of the support task, and that's where the big bucks go. On top of that, one's kids don't want to install their cool stuff on Linux, since it won't run there...
Question from John Meerts, Wesleyan University, CT:
My sense is that the internet has become such an important part of our world wide technology infrastructure that this kind of thing can not continue to go on for much longer. Can you comment on any proposed technology initiatives (not legal ones) that would address this issue and how they may change the way we use the internet?
Gregory A. Jackson:
I don't see any technology initiatives that by themselves make things better. Rather, there are many technologies which, if users use them thoughtfully, decrease risk and, in due course, make the whole system safer. These are mostly things such as I've mentioned earlier: regularly updated antivirus software, host-based firewalls, well-configured email clients, locked-down browsers, stuff like that. They also include mechanisms for identifying sources and senders, such as authenticated software (Microsoft has been very good about this in its updates, for example), various digital-signature schemes, certificate-based authentication, and so forth.
But ALL of these depend on end-users -- and especially recipients of email and visitors to websites -- using them. Strategies that presume that "the managers of the Internet" will somehow stop all traffic that doesn't meet some security standard -- such as the email-stamp strategy Bill Gates has been discussing -- can't succeed, if only because there is no point of authority, and because at least in my neighborhood people keep running stop signs.
Question from Richard Katz, EDUCAUSE:
Greg, Back to your point about IT and the decentralized nature of decisionmaking in higher ed. Do you think we can long continue to maintain as inviolate this fundamental governance. More simply, can we continue to allow those with money to drive where IT dollars go - with all of the security and suboptimization implications that go with this?
Gregory A. Jackson:
The "we" who want not to allow this typically aren't the "we" capable of actually changing things. Things really aren't as bleak as your question suggests -- that is, most faculty, students, staff, and departments are willing to compromise some of their resources and autonomy in the interest of central effectiveness -- and I think they're getting generally better. But I actually believe that part of the essence of higher education, at least in my research-university sector, is the very decentralization and autonomy that makes my job, well, interesting.
Question from Charlie Derr, Simon's Rock College:
What specific (preferably open-source) tools do you use/recommend for network monitoring?
Gregory A. Jackson:
Now you're exposing my ignorance, plus the tools that we use change, and so I really can't answer specifically. (Write me privately and I'll connect you to my security folks, who can.) But we don't use many "free" products (which is what people usually mean when they say "open source"). I know we use lots of Cisco's tools, and some Lancope stuff, and Packeteer for management, but they are part of a broader arsenal. One can't manage networks effectively on the cheap.
Question from Chuck Rothman, Siena College:
We're finding that spyware is becoming nearly as big a problem as viruses on student computers. What steps are colleges taking to address this issue?
Gregory A. Jackson:
I think most institutions have been ignoring the problem, or in some cases recommending installation of one of the respected anti-spyware packages. It's a murky field, though, much in flux.
Question from Ty Brennan, Salve Regina University:
Have you seen any 'best practices' on campuses with regard to ensuring that the hundreds of student computers that arrive in the fall will not contaminate the university network? Thank you.
Gregory A. Jackson:
Here most of us learned from each other last fall. Typically the best practices involve (a) distributing a CD with the most critical patches for known vulnerabilities, and with routines to check the system in broad terms for things like updated OS, (b)having initial network connectivity, at least in dorms, be very narrowly constrained, (c) scanning machines connected to the constrained network for known vulnerabilities, and observing whether they scan outward, have known bad ports open, and only if the answer is "no" (d) authenticating the computer user, recording the associated MAC address, and providing real networking. Antivirus software and so forth too, of course.
Question from Jocelyn Kasamoto, U of Hawaii:
We have a system-wide site license for anti-virus software which we distribute free to our faculty/staff/students. Have any of you deployed personal firewall software to your campuses? If so, do you have any recommendations? We have support issue concerns with these products.
Gregory A. Jackson:
We're just beginning to do this, and yes, there are support problems. None of these products is perfect, but in our view the benefits are going to exceed the costs. We're doing lots of work to preconfigure the clients before we distribute them, so that each user doesn't have to go through the elaborate training routine ("do you want to allow this program", etc). As I recall, in our evaluation Symantec's and Kerio's stuff was easiest for users but hardest to manage centrally, something else whose name I've forgotten was horrible for endusers but great for central management, and we ended up choosing the NAI McAfee product as a reasonable (but quite imperfect) balance.
Question from Tim Schenk, University of Virginia:
Do you see any strong differentiating factors among leading antivirus software packages like McAfee, Norton, Trend Micro, etc? I'd like to read about both desktop and gateway if you have an opinion.
Gregory A. Jackson:
No opinions on gateway at this point, since we don't do much of that yet, and the previous question got the client answer.
Question from Al Quiros, Bradley University:
How do you handle the question of supporting personally owned student computers, while reducing the school's liability with handling computers not university owned. This question comes up when there is a system problem with a personally owned student computer, and claims made that university-required antivirus software was at fault. Certainly, these claims are not typical, but students sometimes have badly damaged systems, which may begin displaying their problems upon installation of some antivirus solution. How far do you feel support should go in those cases ?
Gregory A. Jackson:
If people don't want to protect their machines to our spec, they're welcome not to connect them to our network. That said, we test pretty extensively, and so have had relatively few instances such as you ask about. There was a rash of problems with Microsoft security patches not working properly on non-English versions of the OS -- lots of those arrive with our international students -- but that seems to have diminished as MS recongizes the problem. Bottom line, though, is it's the University's network, and we make the rules. If someone's computer doesn't work under those rules, we can sell them one that does...
Question from Damien Dinh, U. Kansas Med center:
There have been quite a buzz in the intrusion detection community about network and host-based IPS solution which is intended to mitigate zero day exploits. Have you seen any implementation of such tech. in the academia community and if so, how successful (cost/benefit analysis) are they? Thanks
Gregory A. Jackson:
Awright, you've won "stump the speaker". zero day exploits? Maybe I know this by another name...
Comment from Craig D. Rice, St. Olaf College:
I guess this is more of a comment than a question, but in response to Charlie Derr's question on tools, we have found the PacketShaper to have some helpful features. While it's primary purpose is to manage/allocate bandwidth, we also use its internal, command-line functions to examine network flows. We have scripted a few of these sets of commands (happy to share), which has allowed us to automate the identification and disconnection of machines on our network that generate traffic typical of some viruses.
Question from Mary Ann Blair, CMU:
When and how do we turn the corner on this, architecturally, software engineering, social engineering, or any other -wise? Is there a magic bullet under development?
Gregory A. Jackson:
There is no magic bullet. This problem is one of social responsibility, education, dissemination and implementation, and all those other things that open societies do but never as efficiently as we'd like. It's a complicated set of problems, and only a complicated array of strategies is going to get us anywhere. The obviously self-serving comment is that the key strategy is education, and that's what we supposedly do best, right?
Question from Dan Wasson, Northwestern Michigan College:
What do you see as the biggest threat we will be looking at over the next 12-24 months?
Gregory A. Jackson:
We'll see a whole next generation of more sophisticated viruses and worms. Then, I expect, we're going to see more sophisticated use of the resultant zombies -- computer that have been compromised but not used in any way that attracts notice -- to cause economic or social problems of various types. On a very different plane, we're coming to appreciate how much a workable Internet really costs, and we're going to have to figure out how to bear those costs as individuals, organizations, and nations. That could lead to very strange and counterproductive costing and pricing strategies, and those could easily threaten the efficient operation of the network
Question from Karl Bridges, University of Vermont:
Do you have any comment on the role of libraries, with often the most open computing environments on campus, and librarians in preventing these kinds of attacks? Are there things libraries can or should be doing better to help with this problem?
Gregory A. Jackson:
Okay, I'll walk into a firestorm: Libraries can't continue to be as open as they have been in the past, and specifically they can't continue to offer open network access without identifying users. I agree fully that certain kinds of records should not be accessible, and therefore some records shouldn't be kept, but the current argument -- that libraries cannot function unless they allow unauthenticated strangers onto the Internet -- must give way to better balance.
Question from Jo Peyton, University of Dayton:
What is your environment - do your users have local admin rights, workstation security policies, do you own the desktops? We have a large laptop community of students who have purchased the machines, so it is a political battle for us to take control of something they own...any thoughts?
Gregory A. Jackson:
Most of our users own or manage their own machines; not only do they have admin rights, we don't. We have control of about 1000 administrative machines, and there we enforce rights and security policies.
We don't take control of people's own computers. Rather, as I said earlier, we specific rules that users must comply with if they connect to our network. If they don't like the rules, then we encourage them not to connect. More on this at http://nsit.uchicago.edu/eaup
Question from John Gerone, Tulane University:
The U.S. Department of Homeland Security has announced an e-mail alert system that will warn computer users in real-time -- both technology professionals and home users -- of security vulnerabilities, potential impact, and how to mitigate any impending threats. Where do you see the US Govt role in implementing the types of Early Warning Systems?
Gregory A. Jackson:
The authoritative info out there, the better, so I think this is a step forward. I don't think it'll help us much, since we're often the source of the info HS broadcasts, but anything that helps, say, my mother be aware of problems is a good thing.
Question from Dave Kelley, University of Hartford:
We've heard/read about "scanners" that detect computers that are not "up to date" or that don't have anti-virus software on them. Can this really be done without some sort of "agent" running on the scanned computer, or is voluntary cooperation of the user required (I'm thinking of students with personally owned computers in dormitories)?
Gregory A. Jackson:
A scanner typically can see what ports are open on a computer, and can try certain protocols against those ports. Certain open-port and response patterns can indicate that a machine has been compromised; they can also suggest what software is running on the machine. In addition, most OSes report what version they are. Beyond this, as you suggest, one really needs something to run on the machine.
Question from Leslie Maltz at Columbia University:
We are considering an automated means for backing up user desktop files to add to the collection of anti-virus, personal firewall, and push update tools. The goal is to prevent as many problems as reasonably possible, and if all else fails, be able to restore files saved before the systems was infected. What is your opinion on this combination including automated desktop backup?
Gregory A. Jackson:
I think it's immensely wise, and am embarrassed that I failed to mention it earlier. We have such a system handling lots of hosts already, and plan to extend it further. That said, this is hard to do well: bandwidth become a big issue, as does storage depending on retention policy (how many versions of each file? How many copies of Microsoft Word should we back up? etc.). Our tape robot, which started out with one bay, now has four with two grabbers. Big bucks.
Andrea L. Foster (Moderator):
That will have to be our last question. We are now out of time.
Thank you, Greg, for being our guest today.
Gregory A. Jackson:
Awright, folks, it's stopped snowing here in Chicago, so we're done!
Copyright © 2008 by The Chronicle of Higher Education