NEW BRUNSWICK, NJ--For for at least four days, an embarrassing and alarming denial of service attack wreaked havoc on academics and other aspects of university life on all campuses of Rutgers University.
It marks the third distributed denial of service (DDoS) attack this semester, leaving students outraged at their school as it struggles to deal with a variety of logistical problems caused by the outages.
One week after the attack first affected internet service and more, officials at Rutgers and their cybersecurity contractor Fishnet Security, have refused to respond to repeated inquiries from New Brunswick Today.
As we reported previously, the school uses inadequate and outdated cybersecurity practices.
"We are making progress towards restoring all network services to normal operating status and working with the Chancellors to develop contingency plans for online exams," read the most detailed update from the university since the attacks crippled the school's online tools on April 27.
The effects of the attack were still being felt well into a fifth day, on May 1, as several students reported spotty wireless internet access on-campus, and problems with online learning tools like Pearson's eCollege.
Still, the school said that its networks were "performing reliably throughout the day," in a public message.
The university's online sytem for class registration was not working on April 27 or 28, the first two nights of a scheduled registration period, forcing officials to devise a new, abbreviated schedule that began on April 29.
University officials did not answer questions about whether or not the process has been working without glitches.
But the class registration problem is just one of a wide array of inconveniences magnified because there was no contingency plan in place.
"While we are optimistic that our IT teams will resolve this situation soon, we are confronted by the realities of the academic calendar," wrote New Brunswick Chancellor Richard Edwards on May 1, bracing faculty for "the possibility that the network will remain challenged into next week."
Edwards told faculty to consider giving paper exams, noting that online and "hybrid" courses "are even more adversely affected."
"All of the academic deans join me in asking faculty who rely on the network for online exams and final projects to prepare contingencies."
"We understand that this will require scheduling appropriate examination space, and the scheduling office and the offices of each academic dean will make every effort to assist you. You might also consider providing an alternative final assignment, like a short paper," wrote Edwards.
Still, Edwards admitted some online resources could not be easily replaced:
We understand that there are many reasons that you may not choose to substitute with a traditional bluebook exam or alternative assignment. For example students may have to rely upon online resources—such as Sakai and eCollege—to access their course materials, and if those resources cannot be provided, they may not have the information that they need to complete coursework and/or exams. If you choose not to give a traditional bluebook exam or alternative assignment, you can give final grades based on the work that has been done to date. If a student is not satisfied with that result, the student should have two options: the student can receive a "Pass/Fail" grade or take an "Incomplete" and take the exam when it becomes feasible to do so.
In classes that have project deadlines, we encourage you to be flexible and give students an extension when the delay is caused by the disruptions of the network outages, and we ask in particular that you offer special consideration for those students who are poised to graduate this May.
If you believe the proposed solutions will not work for your course, please contact your academic dean’s office.
Please note that the timeline for assigning final grades is still in effect. Should the online grading system be unavailable when final grades are due, a paper alternative will be provided.
In practice, some professors are more accomodating than others. While many students praised the cancellation of exams or deadlines getting pushed back, some were forced to take tests or submit work without online resources that they have come to rely upon.
During a four-day-long cyber attack in March, students were still not allowed to re-schedule an exam for Organic Chemistry, held in the middle of the outage with no option to take or "re-take" the test at a later date.
Making matters worse, at the same time the latest attack was crippling the entire Rutgers network, high school students across the country faced a May 1 deadline to confirm their attendance at colleges for the fall.
Some found it difficult to commit to Rutgers, which moved its admissions process entirely online years ago.
"Rutgers I can't commit because of the internet outage, the website is down," wrote one prospective student on the Rutgers Facebook page. "I need help because the deadline is approaching. Message me back on Facebook please!"
An anonymous person with a Twitter account has taken credit for the attacks, which have affected Rutgers' systems at least four times since November 2014.
Throughout the crisis, the university has issued updates twice daily, via social media, each providing little valuable information beyond the fact that they were working on the problem.
"As of 5:00 PM on Thursday, April 30th, the University continues to experience degraded online services. The DDoS attacks are ongoing and persistent," reads one update, which gave some specifics about the technical response.
"Over the past several days the Office of Information Technology (OIT) has been implementing a multi-faceted approach in response to the recent technical difficulties."
The announcement also listed a numer of measures taken "to reduce and stabilize the issues," including:
"Simultaneously, OIT and the Rutgers Police Department are to investigate the source of the attacks."
"Due to the ongoing investigation, we are unable to share additional details... We appreciate your patience and cooperation."
A spokesperson for Fishnet Security, a contractor hired by the university at a cost of $307,000, refused to confirm whether Rutgers was one of their clients.
"I'm not at liberty to say," said company spokesperson Lauren Howe.
Fishnet recently merged with its biggest competitor, Accuvant, and the resulting conglomerate will soon be known as Optiv Security.
A Twitter account allegedly operated by the perpetrator of the attacks mysteriously vanished from the internet on after the second day of the attacks, only to re-appear some 24 hours later with all but a few tweets deleted.
After the brief hiatus, the alleged perpetrator issued a statement, claiming that Rutgers made a bad choice by hiring a California-based company named Incapsula as its DDoS mitigation provider, saying the firm was "OK for protecting websites, not a university."
Rutgers did not respond to a request to confirm whether or not they had hired Incapsula, and a person who answered the phone at Incapsula hung up the phone on a New Brunswick Today reporter asked for their press person.
Others who answered the phones there promised someone would get back to us. No one did.
Rutgers also did not respond to email and phone inquiries about the company, refusing to confirm that they were doing business with Incapsula.
Below is the alleged cyber attacker's full statement, released on April 29:
The Rutgers IT department is a joke. This is the third time I have launched DDoS attacks against Rutgers, and every single time, the Rutgers infrastructure crumpled like a tin can under the heel of my boot. This is a surefire sign that somebody needs to be fired...
The pure incompetence of the IT department just amazes me. They took a step in the right direction by hiring a DDoS mitigation provider, Incapsula, to assist them with the attacks. However, they ended up destroying connectivity and routes in the process. I did not launch any attacks on Tuesday. Why should I have? The inexperience and poor skills of the IT department ensured that I didn't need to. I sat here watching them foolishly attempt to fix the routing issues, withdrawing and appending routes (there were some instances when I laughed at what they tried to do).
Anyway, good on you for finally realizing that you needed a DDoS mitigation provider! However, I'm guessing you went with a large company who offered you the lowest bid, because out of all the providers you chose, you picked Incapsula over Verisign and Prolexic...
Incapsula is OK for protecting websites, not a university. Honestly, I am sitting here dumbfounded at the amount of incompetence displayed once again by the Rutgers IT department. I'm fairly certain I could run circles around all of you with my eyes closed, and one leg amputated...
Just to show you the poor quality of Incapsula's network, I have gone ahead and decimated the Rutgers network (and parts of Incapsula), in the hopes that you will pick another provider that knows what they are doing. Furthermore, please fire all the people that made the decision to use Incapsula immidiately.
Rutgers said it was "actively working in consultation" with the Federal Bureau of Investigation (FBI) and the Office of Homeland Security and Preparedness to investigate the source of the attacks.
"The FBI is assisting Rutgers in the investigation regarding the denial of service," said FBI Special Agent Celeste Danzi.