Special Topics: Systems Design for Cloud Computing

Not a lot has changed since the previous reviews. I found the described difficulty of each module to be accurate. This isn't a course where you have to check off a list of requirements. You are treated like an adult and I found the TAs always acting in good faith. Overall a great course even though half of it is networking related.

SDN is by far the most difficult mainly because the project hits you like a truck. You think conceptually it makes sense and you should finish it quickly, but the implementation takes forever.

NFV would be difficult if you don't grasp the concepts properly which is a real possibility since there is not a lot of time during the semester. Debugging why a network packet doesn't reach your destination is one of the least fun things in life. If the goal of this module is to reinforce how much of a pain network flow attribution is, it does it very well.

Systems module would be a lot of fun if the word boundary issue didn't exist. Learning kubernetes is nice. Listen to offered advice and stick with Go. I'd rather sleep on a bed of nails than deal with the C++ build system.

The apps module can be as difficult or easy as you want. Typically burnout sets in around this time and most folks choose something straightforward. Unsurprisingly there were many AI-related projects this semester.

--

I personally think the reddit /r/OMSCS mods are great. They were considered gifted when young and had an audience with POTUS himself as photographed here - https://imgur.com/a/ZDag8E7

Despite the review being labeled for 2023, I took this course for the Spring 2024 semester. Just wanted to submit this before I forget.

The topics covered in SDCC are easy to grasp (relative to AOS topics) and the workshops are somewhat useful for exploring what you've learned... but the project work is tough. Really tough. We started with 85 students and had 54 remaining after the withdrawal deadline—nearly 40% of the class. Ultimately, though, I feel the course and its lessons were very insightful and worth doing. This is a chance to learn the foundational concepts that the "cloud" is built on; hugely beneficial if you're someone that is already working on either building infrastructure in a public cloud, involved with designing cloud-like infrastructure for an organization, or writing applications that will be eventually deployed in the cloud.

If you are not already familiar with systems programming or cloud concepts, I'd strongly advise against pairing this course with another—particularly if you have family, full-time work, or a social life and/or hobbies you want to keep up with. You WILL need ample time to read into these topics and various docs to be successful in the course and gain the most out of the course. If you enroll with a mindset of "whatever I'll just grind through the code and GPT the assignments", you are missing the point and wasting your time—even if you pass. If you're not working full-time and are already somewhat familiar with the course topics, pairing with one other course should be fine.

Some other qualities I took note of during the semester:

The Good:

• Prof Kishore is very enthusiastic and eager to talk about and expand on topics
• Subject matter is interesting and relevant—unlike a large portion of AOS, I can see how the tools and course content might be applied in practical scenarios (like at my job)
• The projects are rigorous but offer many learning opportunities
• I appreciate that the course allows for—and even encourages—collaboration and chatting about potential solutions; a fair number of students were active within the Slack channel throughout the term.

The Bad:

• Some of the project specifications and resources are vague, outdated, or incorrect.
• While I understand the intent, I don't think the weekly sessions should be mandatory for passing the class. People in other time zones (sometimes, other countries) are greatly inconvenienced by this and the sessions themselves don't really provide any value—other than students having the option to asks questions live.
• Considering that CS6210—the prerequisite to this course—already has a MapReduce project, it is somewhat disappointing that the same topic is being re-hashed... even if it is a fuller implementation of the framework. Surely there are other interesting innovations in systems software that we could study and implement?! Folks who are auditing or are somehow in this course without taking CS6210 course might be fine with this... but I wish we could have explored something else.
  • Also somewhat annoyed with the fragmented approach. Rather than splitting MapReduce into workshops with odd requirements, it'd be better if we were simply assigned the entire project and given 3-4 weeks to complete it.
• Scheduling for workshop and project grading is kind of rough. TA's share out a (poorly-formatted) spreadsheet and tell teams to sign up for times to present their workshops or projects. There are either not enough TAs to support the number of teams or TAs don't have wide enough availability windows to accomodate everyone.
  • In addition to this, TAs may not post their availability until the day before they are available and may—albeit, rarely—change their availability. This is pretty inconvenient at times, especially considering we are not automatically extended the same grace unless there's some major, documented event (e.g., outage of GATech Github, medical reason, (provable) personal emergency)... at least, that's the tone that's set from the beginning of the course.

The Ugly:

• TAs are basically MIA on Ed and Slack. Maybe they're only answering private posts?
  • Would be nice if they were responsive beyond office hours or the weekly meeting; I understand that this is likely a part-time gig for them but the least they could do is check Ed and respond to a few posts. And even that was barely done.

Just some quick context before I jump into this review: I took AOS in spring of the same year. I work full time as a backend SWE. For this semester, we just finished the mapreduce module as I'm writing this

Compared to AOS, this class is very different. It isn't as lecture heavy, there are no papers, there are no exams. It's as if you took the AOS projects and tripled/quadrupled the requirements. And these projects feel most similar to the libvirt project in AOS: you have to sift through a bunch of documentation and come up with your own design in order to satisfy the requirements. It isn't like they give you a pre-implemented framework where you fill in the blanks, you will have to suffer through a bunch of documentation and start from scratch for a lot of it

Overall structure of the course: you have 4 modules. Each one is split into 4 weeks, first three weeks have a workshop due every wednesday, last week has a project that's due. Every week you present to a TA, normally the project demo is more involved than the workshop demo

I'll summarize the modules now:

SDN: this one assumes some knowledge of networking (high level understanding of ARP, switches/links in a LAN). You don't necessarily need a networking class under your belt to survive here (I didn't), but it might be nice to have. This module is very unique, you use linux's built-in network virtualization to virtualize a network topology of hosts, links, and switches, then use an SDN framework called Ryu to programmatically install traffic rules on the switches

NFV: ditto wrt networking stuff in the SDN module. This module isn't as hard as SDN, but it's no walk in the park either. It also builds on top of SDN, so if you struggled in SDN you'll struggle here too because it builds on top of prior knowledge. It's similar to the SDN unit with the hosts, links, and switches, except now some of the hosts use linux iptables to behave as network functions (e.g. host2 acts as a firewall between hosts 1 and 3). It also teaches you some docker stuff, SDN unit was using a tool called mininet to set up the network topology, in this unit you use docker instead (which is slightly more involved)

Systems: This one does a better job of spreading the workload out across the different workshops. In SDN and NFV the workshops are on the easier side and it ramps up a lot more for the projects. In this unit, you build a mapreduce framework (primarily for doing wordcount because of how they want you to shard inputs). The framework is deployed to k8s. You have to expose external APIs on the master for submitting jobs and deploy it onto azure k8s

Apps: In progress right now. Anyway you choose your own project for this one so YMMV

In terms of difficulty (for me): SDN > NFV > Systems. I write this because I've done a few mapreduce projects. For someone not in that boat, I'd predict something more like SDN/Systems > NFV

Overall: a very fulfilling course. Very SWE heavy, most of the work in this class is spent on the projects. And they are large projects, some of the largest I've worked on in this program so far

Difficulty breakdown: 6/5 for programming, 3/5 for conceptual. Project difficulties: Apps < Network Function Virtualization < Software Defined Networks << MapReduce

TLDR: The concepts in this class are pretty simple. The difficulty is in learning a bunch of new libraries and implementing things nearly from scratch. It's more of a SWE class than CS.

The concepts are pretty simple, so this class is more about burning fundamentals into your brain than anything else. The last two projects are very practical, with the last one being entirely a SWE project with basically nothing conceptual. I learned either directly or incidentally about a lot of tools I didn't know of before like Wireshark, Jmeter, Kuberneted-in-Docker (KIND), and Apache Kafka.

A special note about the MapReduce project since it's by far the hardest. In hindsight, handling dependencies with C++ is a shitshow and it would've been easier to learn GO from scratch. Heck, even writing Map Reduce in entirely Python is probably more viable than C++ even though the starter code offers only C++ or GO and it lacks true multithreading. I recommend sticking to a language with real package management unless you're a pro at using cmake. C++ development experience doesn't count if all you did was code in it, and no you will not be the genius exception who learns it easily without wasting days on troubleshooting.

If you still don't believe me and do try to use C++, don't use the starter code. Just make a list of dependencies and install them yourself.

The best summary came from a classmate in our last synchronous class meeting - "This course is the most fun I never want to have again."

This course is a staggering amount of work. It is also incredibly valuable. It's perhaps not a stretch to say that it's entirely possible to get a job solely because of this course. Prof. Kishore's network is very strong, and recruiters know that this course speaks to tangible skills in high-demand areas.

At it's heaviest, taking this class was like having a second, very intense job. I'd say that during some of the big pushes to complete projects, I spent close to 50 hours on work. Full days both Saturday and Sunday. Plus a substantial part of the week. I'm lucky to have a fairly flexible or I'd probably have needed to take days off to finish up projects. Even though you're working in a team, it can be very hard to split up the tasks. My team ended up splitting more along project lines - I did most of the MapReduce project while my partner did most of the final project. So on balance, it was pretty even, but often quite stark in the workload between teammates on a per-project/workshop basis.

You will come away with a deep and practical understanding of the material though. SDN, NFV, and how to build an effective, fault-tolerant, distributed application framework in the cloud.

N.B. - it helps a lot to have a very functional MapReduce framework built from 6210. My framework from 6210 was about 1500 LoC and had a lot of the nice "optional" features around fault tolerance. I more than doubled that for this class to handled running it in Kubernetes. I can't imagine the teams that started from scratch with Go. We used C++ (building off my 6210 framework) and then etcd. This was definitely the biggest project, but not by much. The SDN and NFV projects were also substantial. Having real expertise with both Python and C++ is a requirement for this class. Also, don't even think of doing anything else (TA'ing and/or taking another class).

The in-person classes (8:30 - 10PM Eastern which was great for me being in California) were fantastic. The TA's are all superb. And Prof. Kishore is, without question, the best professor I've ever had for any class ever. His passion and enthusiasm for teaching and learning are infectious.

There are only two grades - A or F. And the teaching staff says that you have to work hard to fail, and they are right. But you also have to work hard to "pass" as well. They make you earn the A, but they also go above and beyond to make sure you do.

This is, by far, the best class I've taken in this program (my 4th), and I don't expect that to change. This is what OMSCS is all about. Incredible learning opportunities that only really can exist in this way because of the incredible nature of this program. I loved this class. But I'm also sure glad it's over with. If you think you might want to take this class, do it. If you're on the fence, take a pass until you're sure. It's too much work, and the team-based nature means you not only affect yourself but a teammate.

I loved this class and cannot say enough good things about it. But it's also imperative that you know what you are signing up for.

Short Summary

This was easily one of the best courses I have taken so far. It covers a lot of topics which I found to be extremely relevant in today's industry. While it's a Be sure to find a good teammate!!!! I cannot stress this enough :)

Modules

The course is divided into 4 sections:

• SDN: The workshops in this one were pretty straightforward, but you definitely need to spend time to understand the workshops at a deeper level. If you don't, the project submission week might get a little rough depending on your previous experience with SDNs. Even though my teammate and I started a week earlier, we ended up working till the last day trying to iron out small bugs.
• NFV: This one was a little simpler, with the project being a small extension to the knowledge you gain in the SDN module. Effort will more or less depend on how comfortable you got with the SDN module.
• Systems: 4 weeks of constant grind :D. But you learn a LOT of useful skills! If you haven't taken AOS before, you might be at a disadvantage because it's MapReduce on steroids, from scratch. You can use part of your AOS code, but the reusability is limited.
• Cloud Apps: This one was pretty chill, and you can truly feel the proximity to the end. The workshop content is alright, and you mostly learn at your own pace, depending on how much effort you put into the project.

Workload

For background, I did not take AOS myself, but familiar with Python/Golang/Kubernetes. And in this case, the system part is ok for me, like it would take 10-15 hours a week (also it seems easier to implement it in golang compared to the implementation of my teammate in C++). But still, if you're not familiar with them, it would take 40-50 hours as others mention.

The hardest part for me is the first module SDN which takes 20-25 hours per week (I do not have strong network background and also the SDN documentation is not so detailed lol). For the second module NFV, workshops are good and take around 10 hours and the project takes 20 hours. And the last module to build a cloud app yourself would be relatively easy but still considering what the project you propose to do.

Recommendation

I should say I have quite a lot of struggles during the course but still it's a great course to get you to dive into different areas of knowledge and projects. And remember you do not want to take another course with this size of workload in the same semester.