It’s summer here, another semester at CMU flies by. A lot happened in this semester professionally and personally for me, so I will try to keep the update brief and highlight the relevant points.
I started my semester with 4 courses (48 units), 1 TAship and an on-campus job. The courses I started with were 11785 (Introduction to Deep Learning), 11747 (Neural Networks for Natural Language Processing), 18898D (Graph Signal Processing and Geometric Learning), 18980 (MS Graduate Project). Along with that I TA’ed for 18661 (Introduction to Machine Learning for Engineers). Overall it was very hectic, but I was doing well in the course and assignments.
Things started to slip around mid Feb, when I had to go to Seattle for an internship interview with Microsoft. The whole process of flying to Seattle, a full day of interviews followed by flying back took 2-3 days of my time. This made me realise I can’t even afford 2-3 days of slip-up otherwise I would be back to scrambling assignments without maximizing learning. On the other hand, my research was picking up speed and something I wanted to devote more time to. Considering all the options, I decided I would drop course 11747 (N3LP)
11785: The course covers fundamentals of deep learning. The content starts off with MLPs, then moves into CNNs, RNNs, sequence-to-sequence models, Attention, Autoencoders, VAE, GANs. The content is covered in depth and details. To enforce this it has four total assignments focusing on MLP, CNN, RNN, and Attention. Each has two parts: In part 1 you are expected to code everything from scratch in Numpy without the use of any automatic differentiation library like Pytorch. The intention is to understand everything from ground up. In part 2: you are expected to use Pytorch and compete on Kaggle by pushing the models in terms of performance. This covers the more practical aspect of making models work. There are 14 timed quizzes in total which need to be completed over the weekend and a Final Project. Our final project was using MTL motivated segmentation auxiliary task to improve relative monocular depth estimation. The proposed framework beats the existing benchmark by 6%. The final project video can be found here. The course website can be found here.
18898: The course covers Graph Signal Processing and Geometric Learning and is a mathematically intensive course . This course will present a novel data analytics perspective to deal with data supported by graphs. Such data occurs in many application domains from traditional physics based signals like with time series, image, or video signals to data arising in social networks, marketing, corporate, financial, health care domains. The first half explored how to extend traditional Digital Signal Processing methods to data supported by graphs (Graph Signal Processing) and the second half focused on how to modify the structure of deep learning models to reflect the underlying data geometry (Geometric Learning). The course had 5 lengthy and mathematically intensive assignments which covered all aspects from GSP to Network science to Graph Deep Learning. The course also had a final project component. Our project was Time Context Aware Graph Convolution Network (TCA-GCN) for Traffic forecasting which extends GCN to take temporal dependencies into account and outperforms benchmark by 7%. Details will be released soon
18980: I did my MS Graduate Project under Prof. Bhiksha Raj on Weakly Labelled Sound Event Detection. My work extends the traditional Multiple Instance Learning (MIL) formulation of WLSED into a more super-set general framework. The work also introduces a novel 2-step attention mechanism as an improvement over traditional pooling. The proposed approach outperforms benchmark models by roughly 22%, 12%, 6% on 0, 10, 20 dB SNR audio respectively. The details are to be released in a paper soon along with code.
I TAed for 18661 taught by Prof. Gauri Joshi and Prof. Carlee Joe-Wong. I had fun creating HW, conducting recitations & office hours and final exams with fellow TA’s. It was my first proper TA experience, and made me release the difficulty and the amount of invested effort in making conceptually challenging yet approachable questions and exams. Amidst COVID pandemic, the class shifted online. The challenges the online format brought in terms of conducting office hours and setting exams and the way the team shifted overnight to the changes was amazing! But TA work though fun is work nonetheless and takes a good amount of time out of your routine. Looking back, TAing is something I might do in the near future, though not in the next semester :)
CMU team’s effort to tackle COVID:
I was a part of CMU’s team headed by Prof. Rita and Prof. Bhiksha to build an automated AI system to detect signatures of COVID 19 infections in human voice. My contribution here was building an automated pipeline and Neural network based model that could without supervision detect coughs with 1 sec precision in audio clip of any length and store it in ordered format. This processed cough is later used to train multiple algorithms. Given any wild audio clip recorded from any device, the system delivers the similarity of the voice signature to previously analyzed COVID 19 voice signatures. You can donate your voice here to help make the algorithm better: website. The work has appeared in multiple news, some coverage: Forbes, dailymail UK, Android Authority, Gizmodo.
Note: Take the results with a grain of salt, they are not conclusive medical advice. Please visit the doctor if you don’t feel well.
Microsoft Internship interviews:
In the first week of January, a recruiter reached out to me to interview with Microsoft for a Data and Applied Scientist Intern position. I barely had any preparation time due to an already hectic semester going, but trying couldn’t hurt right? So I went ahead with it. The first phone interview was scheduled with a Data and Applied Scientist at Microsoft. It was scheduled around 15 Jan, and was about 30-45 mins phone interview about general DS topics and resume. Bit ashamed to say, at the last moment I had to shift the interview by 15mins as my TA meeting ran over scheduled time. I cleared that round and was moved to the second round. The second round is an onsite full-day interview round at Redmond, Seattle Microsoft HQ. This was scheduled for me around 15 February. I debated the decision of whether to attend or not, the hectic schedule was already burning me and removing another 2-3 days from routine would be a killer. In the end, my friend convinced me to go and looking back it was the correct decision. This highlights the importance of having great friends surrounding you, who care about you and your success. The decision though had consequences, this resulted in me dropping 11747 in the coming week and existing a competition that my team and I had participated in. However, this in future, left me with time to contribute to COVID project when the professors called. Life works in mysterious ways, eh ?
Back to interview. I flew to Seattle on the 10th and had a return flight to Pittsburgh on the 11th night, so that I could conduct office hours on 12th. After these 3 days, I realized traveling is hectic and should never be done back to back with interviews, it simply just drains you. Microsoft had arranged a pick from the airport and a nice stay at RESIDENCE INN REDMOND MARRIOTT. To be honest, it was an awesome stay for a student. On the 12th Morning, I left for Microsoft from the hotel. Apparently they have an entire building for taking interviews. I met other students there interviewing for intern and full-time positions. We had lunch with the recruiter and then full-afternoon of interviews. I had 4-5 about 45 minute interviews with 10-15 min break in between each. The range of questions varied from data structures and algorithms to latest DL algorithms. The questions were very practical, only in one instance did I have a question where I had to prove something mathematically. I interviewed with the Yammer team with another data scientist from Outlook team. Overall, the process was very smooth and every person was passionate about their role at Microsoft and their contribution. This tells you a lot about the value and culture of the company and team. The same night, I went back to Pittsburgh. The interview seems to have gone super well, I received the offer in one day on 12th February. I loved the team, the work they were doing and their excitement towards it. I decided to accept the offer in the upcoming week.
I continued my last semester’s on-campus job for this semester, though with reduced time. The on-campus job was later cancelled around April, when due to COVID CMU shifted to online instructions.
Overall, It was an exciting semester with lots of uncertainty and COVID popping up midway. Some photos to go with it:
Attaching some photos below:
Working on some problem:
Dinner at Redmond MARRIOTT: