Information Security Lab: System and Network Defenses

Lectures: average. The first half of the course lectures are mostly text modules. The second half is mainly repurposed videos from IIS and NS that are sometimes missing important context because it’s not the full sequence of videos from the original lecture they’re pulled from. In either case, the lectures are fairly surface level, and you're not going to absorb much of the material that isn't covered in the projects.

Quizzes: 10 reading check quizzes that are mainly there to ensure you went over the lectures throughout the semester. Most of the quizzes had at least one poorly written question whose wording could change the correct answer depending on your interpretation. Students would often argue with the TAs over wording earlier in the semester, and then I think gave up later on because the quizzes are only worth 10% of your grade. These should be rewritten to remove ambiguity and actually require critical thinking rather than Ctrl-F ability.

Projects: 7 projects that ranged anywhere from 10 to 50+ hours of effort. Like with many courses, this is where the real learning occurs. Grading was fairly lenient, but the TAs could’ve been more responsive to student questions about the projects. Two of the TAs rotated as the POC for each project, which basically meant you were dependent on a single staff member for answering questions and resolving any issues that came up.

Binary exploitation. Probably the best project in terms of no ambiguity in what you were supposed to do, and totally self contained (no reliance on an external server). Malware analysis. A more tedious project, because much of it involved sifting through tool output to find the malware’s dispatching function, and to classify malware behavior depending on what Windows APIs it invoked. The third malware was quite difficult to complete because the Windows 7 VM provided was so outdated and resource constrained that many of the modern reverse engineering tools wouldn’t run on it (assuming they could be installed at all). Android rooting. Pretty plug and chug as there’s a lab guide you follow. System IDS built using a Linux kernel module that hooks system calls and looks for anomalous system call sequences. Quite interesting, but you may struggle if you lack C proficiency or can’t navigate a largely undocumented codebase (the Linux kernel). Network IDS. Builds off Lab 4, with three different machines playing the role of adversary, victim, and analysis sandbox. You write a few daemons to transfer files between machines and try to detect malicious binaries. Web security. This one was pretty unrealistic in that that attacker portion of the project involved finding a web form field that would accept and execute arbitrary shell input. The biggest annoyance with this project is that the external server went down for 3 days over a weekend, and it took that long for a TA to finally reboot it. We did get an extension, and this project wasn’t particularly difficult, but the downtime was extremely annoying. ML for security. You run a bunch of tasks with various parameters to train and test DL/ML models for detecting malware, but the course doesn’t provide you enough ML background to be able to make sense of what you’re doing and the results. Like with Lab 6, you’re dependent on an external server for this project, and it went down a few times. Other times, students would submit broken jobs to the server, which would effectively block the available workers until a TA kill the hanging jobs (or rebooted the server).

Exam: better than the quizzes in that the questions weren’t worded poorly, but a few honed in on minutiae that really wasn’t an important part of the course. If you didn’t happen to memorize that particular fact while studying, oh well.

TAs: they were somewhat absent outside of grading and answering Ed questions every once in a while. I think they could’ve been more responsive to student questions and ensuring that the project resources in Lab 6 and 7 were actually accessible. There were three TAs total, but only two were assigned to lead the projects. The third seemed to only do grading.

Extra credit: you could get 5% by taking an optional exam based on supplemental lectures by Professor Lee, another 5% by offering suggestions for how to improve the projects, and finally another 5% for completing some amount of the NSA Codebreaker challenge.

TLDR: Good course overall with some frustrations about clarity of project instructions and response times from TAs.

Overall, I did enjoy this course and its material. The seven projects cover a wide swath of cybersecurity topics to include malware analysis, web exploitation, intrusion detection systems, android "malware," and machine learning (among other related topics). For the most part, the projects were not that difficult once you were able to understand what was actually being asked. To be honest, the difficulty of this course overall came from often unclear project directions and the unfortunate delays in TA answers to student questions (which had fairly significant impact given that I took it during the shorter summer semester). But, to give credit to the TAs, they often did listen to student feedback and pushed back due dates for projects multiple times when asked. The TAs also made some grading adjustments for this semester that I think were very helpful. Basically, I think the TAs overall really wanted students to succeed, but they were likely overwhelmed with all the work given that there were only two of them. This course could really use another one or two TAs to help out. I think that alone would greatly enhance the course and help reduce turn around times with answering student questions (for the summer term at least, perhaps its not an issue in a normal term with more time). This course also had extra credit in the form of providing feedback on how to improve the projects which really pushed worries about overall grades out of mind. The TAs even made a comment about how the course was truly focused on students learning new concepts through the projects and that no one should stress about grades so long as they put in effort for each assignment.

Be wary of the ML project though if it is still kept around. The one available server had several technical issues with processing submitted jobs that seriously delayed progress for a lot of students. Start that one as early as possible. It isn't difficult, but you can't really control how long it takes to complete most tasks.

This class was horrible structured. And it's a shame because the content is great. There was just a slew of technical issues with a lot of the projects. The TAs don't respond on the forums at all and if they did it was a couple word answer that wasn't helpful. The positive thing is that they graded easily, but it was still an extremely difficult class. You'll learn a lot but at the expense of your sanity.

The projects were structured terribly. They basically all had a tutorial that walked you through exactly what you needed to do but then left out key details that you needed to even understand what you were responsible for. One of the worst structured classes I've ever taken.

lab1 - buffer overflow, which in my opinion was the easiest and didn't have too many issues, it was just structured weird where they stacked all the difficult issues into one with no lead up in difficulty at all. semi-difficult for the average person but nothing horrible unless you don't know assembly/buffer overflow attacks

lab2 - malware analysis, decently easy if you are comfortable with assembly, but good luck understanding angr, it was very weird and confusing and the TAs only explained things in a weird abstract way. Very hard for most of the class

lab3 - android rooting, once again, they didn't explain much but overall the easiest lab

lab4 - host-based IDS - 75% of it is just hooking syscalls then trying to determine if binaries were "abnormal". It really didn't make much sense because of the background noise in linux was just not explained at all and they didn't tell us how to handle it so it ended up just being a mess. Fairly easy

lab5 - network-based IDS - gave us the solution to lab4 to detect issues in binaries sent over ftp. Wouldn't be bad if ftp wasn't old and deprecated in modern day. Just felt too hypothetical and not useful at all. Pretty sad because it's a good concept. Somewhat difficult but nothing too bad

lab6 - web security where a vulnerability was "recently" patched so they had to scrap a whole section that wasn't possible where I think the TAs just were too lazy to research beforehand and just carried this over from many years ago, overall pretty easy

lab7 - worst thing I've ever done. It was supposed to be a ML project to learn malicious malware based on windows internals that malware tried to access. The server broke many, many times. It's designed horribly internally by somebody at tech. I ended up using this as my dropped project and not doing it since there was so many technical issues. They, 2 weeks to the end of class, announced a dropped project

This was probably the most challenging course I've ever taken, across all of undergrad and through 4 masters-level courses here at GA Tech. That's not entirely a bad thing, as this program is meant to be rigorous, but this course suffered from many of the issues that first-time offerings have, notably that the projects were not quite as refined as they otherwise could have been, and many required a key "aha" moment similar to other classes.

The majority of the grade (90%) was based on 7 projects, with a final exam comprising the last 10% of the grade. Some of these projects were reasonable; others, less so given the time.

Info on those 7 projects follows below - in general, each was supposed to be 2 weeks, but COVID-19 led to some changes later in the semester:

1. ROP Chains. Exploiting an application to read a flag using Return-Oriented Programming. This project was challenging, but well-done, building nicely on the overflow attack from IIS. ~20 hours.

2. Malware analysis. Again, this built (to some degree) on the malware analysis project from IIS, but with a focus on activating and analyzing malware behaviors both statically and via concolic execution. 2/3 of the project was doable, the last binary could have used more guidance. This was the project the class struggled with the most. ~60 hours.

3. Kernel IDS. We had to hook system calls and determine whether or not the actions performed by a binary were anomalous or malicious for different inputs. There was some confusion over alerting on just anomalies, or just malicious behavior. The base concept was workable, but some specific requirements (notably that they wanted anomaly detection to be done via some sort of syscall tracking rather than exempting known good) made it so that this would be very difficult to accomplish in 2 weeks. This was the second-hardest project. ~40 hours.

4. Network IDS. This takes the kernel IDS from project 3, and uses Snort and daemons to capture and analyze binaries sent over the wire. Neat project, relatively doable (as long as project 3 went well). The open-endedness on how this was done was beneficial for this one; it didn't matter so much how we got to the end result as long as it all worked. ~25 hours.

5. Exploiting Android Webview UI. This one had potential, and wound up being relatively easy. We initially had to exploit flaws in Webviews via two mechanisms, and then implement a solution within an application to mitigate those vulnerabilities. Unfortunately, one of the two flaws was no longer working at the time we did the project. This took what would have been a 5-10 hour project and turned it into a 40-50 hour project. If the second exploit was doable, it probably would have been a 15-20 hour project. As it stands, ~40 hours.

6. Rooting an Android Device. This is a guided project and lab write-up, requiring less coding. As long as you read through the documentation a couple times, it's not difficult. ~15 hours.

7. Machine learning classification of malware, and mimicry attacks. This was originally a 5-part project, but was later cut to 3. Originally had a one week window, but was extended to about 10 days. Could have used more guidance on this, and there were a number of technical issues. ~30 hours.

The class did end up being curved to an 80 for an A, a 60 for a B, otherwise C. I wouldn't expect such a generous curve in future semesters. I think the biggest issue was with projects being too large in scope. If this had been cut to 5 or so projects over the course of 15 weeks (3 weeks/project, maybe less for easier projects and more for harder projects), it would have been more reasonable. Others have mentioned recycled content, which is a fair issue to raise, but I think most of the learning in this class came out of projects, and that was fairly reflected in the grading. I really hope they get some of the issues with the first run of this class ironed out, as I definitely learned a lot, but there was much more frustration than there needed to be.

This was the first time this course was offered at Georgia Tech. With the honour of being amongst the pioneers a.k.a guinea pigs, I would highly recommend taking CS 6262 (Network Security) as a prerequisite. CS 7641 (ML) or CS 7646 (ML4T) are optional prerequisites, either one will be required to tackle one of the projects that entails machine learning.

There were a total of seven projects:

1. Return Oriented Programming (ROP): Primarily a CTF project.

2. Malware Analysis: You'll be given three malware samples; two known and one unknown and you're expected to assess the samples using both static and dynamic analysis.

3. Kernel Hooking: This task entailed writing a loadable kernel module (lkm) in linux to differentiate malicious from benign activity from a binary sample.

4. Network Intrusion Detection: You are expected to use SNORT in conjunction with shell scripts and a custom daemon to detect both malicious and benign binary files.

5. Exploiting Android Webview UI: Write a javascript code that secretly renders a malicious web page onto the Android browser.

6. Rooting Android: Build over-the-air (OVA) updates to gain a root shell on a locked android device. The entire project was conducted using the Android VM.

7. Malware Detection using ML: You'll use MLSploit and devise a machine learning model based on extracted static malware features in order to detect an unknown malware sample.

The projects constitute 90% of your grade with the remainder of the grade from a cumulative final exam. Expect a grading curve with the cutoffs for this semester being:

A: 80+

B: 60-79

C: 0-59

On the contrary, this course had several teething problems my primary dissatisfaction being vague and unclear project requirements with the TAs constantly pestled and sought for clarifications.

All in all, I give my thumbs up as the projects were challenging and if you had an open mind you'll get to learn much.

This was the first semester this class was taught. Never be the first cohort for an OMSCS course.

The lectures are non-existent (it's all recycled videos from IIS and NS). 90% of your grade are labs and they are ok but highly variable in quality. Some labs are easy the other labs are hard but not in a good way. The instructors sometimes try to insert gotchas so students can distinguish themselves and everyone loses points for lack of clarity. If the entire class misses an objective on a lab I'm pretty sure it wasn't hard but just ill conceived. For the sake of future students I'm hoping this was due to being the first offering and the labs and content delivery matures. It's basically the result of Wenke's IIS and NS class having a baby with Taesoo's CTF class (binary exploitation). Content and style more like the former (same instructor), emphasis on labs like the latter.