The Secret Lives of Millennial CS Assistant Professors (Part 2)
Read Part 1 here: https://arunis100.medium.com/the-secret-lives-of-millennial-cs-assistant-professors-part-1-41dc712375ca
In Part 1, I cover Research, Funding, and Collaborations; Dissemination, Community, and Service; Advisees and Colleagues. This post covers Teaching and HDSI; DEI and Outreach; Personal Life and Health; and other overall observations, including things I wish I knew earlier, mistakes in hindsight, and things I am still uncertain of as I look toward the future.
Teaching and HDSI
Teaching is perhaps the single most important differentiator of academic life. Profs teach various audiences: their school’s students, external mentees, their research community, and even national and global research agencies. As Asst Prof, I focused mainly on UCSD students and my research community.
My regular load at UCSD is 3.5 courses per year, where the 0.5 is a non-lecture area seminar. I teach the following 3 courses in steady state. All 3 were new to UCSD, and I created 2 from scratch.
- CSE 132C: Database System Implementation
- CSE 234: Data Systems for Machine Learning
- DSC 102: Systems for Scalable Analytics
UCSD CSE did not have a systems-focused UG DB course until I joined. So, I created CSE 132C, a senior-level UG course. Its syllabus is largely based on Wisconsin’s UG DB course that I taught in Fall 2015. I created all materials and slides myself — it was a lot of work! — but I based them on my advisors’ slides. I also reused their C++ programming assignments. I made some n00b mistakes when I taught it at UW-Madison: made some exam questions too contrived (needless stress for students) and pushed deadlines too often (needless confusion for students). The student feedback helped me improve. That experience and the reuse of materials reduced the time it took, and anxiety I had, when starting at UCSD. So, I’d advise all PhD students aiming for academic jobs to get relevant teaching experience.
I also taught CSE’s grad DB systems course (CSE 232A) twice. I mostly reused my CSE 132C materials but added two new topics and paper reviewing for extra credit. This class had ~200 students! It led to more bookkeeping, but I did not find it much harder to teach than the ~40-student CSE 132C thanks to our staff and my TAs.
I also created and teach what is now CSE 234, one of the first courses in the world on ML systems. This was/is a lot of fun to create, teach, and maintain. Some students who take it have real-world ML engineering experience, which leads to awesome discussions. I learn new things every year! All my lecture videos and materials are freely available here. Sharing knowledge openly is part of what makes teaching gratifying. I have received kind thank you notes from people around the world (Africa, Asia, and Europe) and in industry for making those lectures/materials freely available.
CSE 234 also enabled me to tightly couple my teaching and research. Course research projects are a low-risk vehicle for extending papers, exploring “out there” ideas, and recruiting new advisees. In fact, 4 such projects led to top-tier papers and I recruited some MS students as advisees. Some continued to do a PhD, in turn becoming TAs for future editions and evaluating paper reviews, which helps their PhD training as reviewers — a virtuous cycle! Overall, my research, teaching, and advising worked together harmoniously, cross-pollinating and strengthening each other.
Lesson: Creating new courses is a lot of work but it can be rewarding and impactful. Do so prudently to fill key curricular gaps and bring structure to an area. Use course research projects as filters to recruit advisees.
Perhaps the most work I have done on teaching is for HDSI, a from-scratch “department” of Data Science UCSD launched in 2018. Read this interview for why I think investing in Data Science is crucial for academia. HDSI has a carefully crafted BS curriculum with some new courses that neither CSE nor Statistics/Math have. I co-defined a 3-course series on data systems: DSC 100 (Intro. to Data Management), DSC 102, and DSC 104 (Beyond Relational Data). The first 2 are required courses.
I created DSC 102 from scratch. AFAIK it is the first course of its kind in the world. It gives Data Science juniors a unified view of the systems stack from bottom to top — computer architecture, OS, cloud, and scalable/parallel data systems — but only what is relevant for Data Science workloads. It was a challenging but exciting pedagogical exercise to read textbooks in 3 CS areas (comp. arch., OS, and DBMS) to extract, synthesize, and create new content in a way that is suitable for (future) data scientists.
Thankfully, my 3 senior PhD students — Supun, Vraj, and Yuhao — helped out as TAs and co-created challenging programming assignments to analyze large datasets with Dask, Spark, AWS, and SDSC private cloud. UCSD ITS/DevOps staff said my course was challenging to administer but rose to the occasion. In the end, DSC 102 consumed a massive ~25% of UCSD’s quarterly AWS budget, ~10x higher than the next course! I guess you can call me a “10x professor” now. :D Nevertheless, all this taught students the importance of budget-awareness, a crucial skill in real-world Data Science. It also led to some hilarious memes in the second edition. :)
The first edition went well even though it was a large class (~150 students). I collected feedback on the content and delivery thrice. But just before finals week, the COVID-19 pandemic hit the US. :-/ I had to scramble to figure out online exam options. Online proctoring solutions seemed unscalable and too intrusive. Even worse, I got word of a cheating-collusion plan involving 30+ students via whistleblowers! :( I take academic integrity very seriously. So, I decided to crack down on such indiscipline.
After speaking with some HDSI colleagues, I enforced all Canvas features to thwart collusion. The inevitable tradeoff was that the exam logistics became harder for everyone. ~80% of my class agreed with the new policy but the rest were unhappy, leading to polarized teaching evals. I took it in my stride. While teaching evals are indeed important for Asst Profs, one must filter noise in student feedback lest the baby is thrown out with the bathwater. I was not worried because all my prior teaching evals were strong anyway.
After the quarter, I interviewed a few diligent students to collect more specific feedback. I also took the “course design studio” offered by UCSD’s awesome Teaching+Learning Commons to apply some best practices from pedagogical research. All that led to changes in both the lectures and the programming assignments, streamlining DSC 102. A couple of HDSI alumni who took up data scientist jobs later told me that DSC 102 was the “most challenging” but also the “most useful” course they took at HDSI! Hallelujah! :)
Let me conclude this section with some more policies I’ve found helpful. I use absolute grading scales in all my courses. I find them more transparent and fairer than “grading on a curve.” They also seem to spur more hard work and make the class more collegial. Setting absolute scale expectations is not easy but it is feasible with precise learning outcomes and tailoring evaluation components accordingly. I use Bloom’s taxonomy to create a suitable mix of evaluation components and also exam questions. My in-person exams are closed notes/Web; online ones, open notes/Web. I grade all exams myself for consistency on partial credits.
Lesson: Academic integrity is crucial; enforce without fear of teaching evals. Separate wheat from chaff in (noisy) student feedback. Use precise grading criteria and learning evaluation rooted in pedagogical research.
Finally, I also did some “teaching” aimed at my research community. I co-presented a well-attended tutorial at SIGMOD’17 on research in my area. That led to an invited textbook, the first such book on ML systems. It is unusual for CS Asst Profs to publish a book, but I saw it as key “teaching service” for my community. I also joined this fun panel discussion at VLDB’21 on the future of DB education, sharing about my experiences with all my courses, HDSI, and more. I hope all these efforts are helpful in shaping the future of DB and ML systems education in both CS and Data Science.
Diversity, Equity, and Inclusion (DEI) and Outreach
One of the things that made UCSD appealing to me even before the interview was that they asked for a “contributions to diversity” statement from all applicants. Prima facie, that signaled to me that UCSD took diversity and inclusion (D&I) seriously. People from marginalized groups often have basic questions about a potential employer they may never voice: Is it a safe place to work? Is the culture inclusive? If I face a discriminatory event, will I be supported or ignored/gaslighted? As part of writing my statement, I looked at UCSD’s D&I policies, initiatives, etc. All that inspired me to think about what I could do as Asst Prof to help improve D&I in CS, whether or not I ended up at UCSD. I learned later that UCSD was the pioneer on this front, inspiring many other universities to ask for such a statement too!
But why do I care about D&I in the first place? As a gay man myself, I know first hand how deeply dispiriting discriminatory experiences can be for people from marginalized groups. Such experiences often force people to leave their job or even their field. Most people from majority groups have the luxury of never facing such issues. I also know how representational diversity in visible roles in society can help raise self-confidence in kids from marginalized groups. So, I consider it important to help make my field accessible and inclusive to people from all backgrounds.
From a deontological standpoint, it is basic human decency to enable people of all backgrounds to actively participate in (and benefit from) our field. From a utilitarian standpoint, diverse workforces can help organizations combat groupthink pitfalls and be more creative. Later I also learned of the role of equity in mitigating the effects of both past and extant discrimination and marginalization. In a larger philosophical sense, DEI goals operationalize the timeless principles of justice and truth. Being a Tamil and an Indian, these themes were familiar to me, e.g., from the great Tamil epic Silappathikaram, this famous Upanishadic verse, or Mahatma Gandhi’s life.
Alas, a long history of systematic discrimination by institutions and societies has yielded many cruel legacies that we must surmount. I have written about such legacies in the US and India in this blog post. Sadly, many people in CS, especially in academia, seem oblivious or consciously callous on such issues. Some think they are just being “objective” but to me they seem more like racehorses with blinders, failing to see the bigger picture.
Anyway, I do not consider myself a DEI expert, but I am glad that both CS academia and industry are now having these long-overdue conversations. It also helps that NSF CISE showed leadership with their pioneering Broadening Participation in Computing (BPC) initiative. I hope all this momentum continues and leads to actual positive impact.
I will skip details on all that I did/do as part of my DEI work (see pages 12–13 of my CV). But at a meta level, I’d advise Asst Profs to choose activities based on what is meaningful to you, potential for impact, visibility, and effort/time required. In my case, I balanced the following 3 types of activities; each type is important but the amount of personal effort required varies.
- Directly mentoring students from marginalized groups.
- Systemic efforts for my department/university/city.
- Systemic efforts for my research community.
As an example of type 1, I work a lot with the UCSD LGBT Resource Center and UCSD oSTEM to help LGBTQ+ students. For instance, I joined many Q&A panels, annual events, and “Rainbow Graduation.” I hosted lab open houses and even recruited a student researcher to my group. I also host strong interns from SoCal each year: from Hispanic-Serving community colleges via STARS and local high school girls and boys via MAP. I am glad I was able to help some of them with their college/grad school applications and/or job search. I’d advise all Asst Profs to leverage such programs at your university. They are the simplest and most direct form of DEI contributions.
As an example of type 2, I co-represented UCSD CSE at NSF’s BPC Workshop, co-authored our Departmental BPC Plan with the Chair, took inputs from the DEI Committee, and got it approved by NSF. I also gave inputs on HDSI’s BPC plan and served as a founding faculty member of the HDSI DEI Committee for a year. All this is systemic work to help my UCSD colleagues. Finally, as an example of type 3, I helped SIGMOD’21 as an inaugural D&I Co-Chair: see this article and this tweet on our work. I co-launched the D&I in DB Initiative (likely a first for all CS areas) and helped them early on.
A common pitfall in DEI is to “dump” such work on people from marginalized groups — the so-called “minority tax” — because it is (mistakenly) assumed such folks will know more. Thankfully, there is growing awareness of that issue. It is also great to see more folks from majority groups stepping up as allies to further DEI efforts in CS, speak out publicly against major discriminatory or exclusionary events, etc. I hope this momentum continues. I also hope academic incentive structures evolve to value DEI impact. For example, UCSD often rewards strong DEI impact with salary bonuses, subject to strengths on the regular criteria, of course.
Lesson: If you are troubled by inequities/discrimination/marginalization of people in CS, step up with DEI service. Do not sit on the sidelines. Reach out to relevant people. Educate yourself. Choose activities that work for you, ranging from short-term student mentoring to long-term systemic efforts.
Of course, not all DEI ideas may pan out as intended. Some may even end up (inadvertently) counter-productive. We need earnest debates to weigh risks vs benefits and avoid pitfalls, just as we do with research projects. Continual vigilance is crucial. Also, not all people may care about DEI; some may even be opposed. To me it is important to respect their right to express dissent. But I hope such folks have the prudence to voice criticisms constructively to avoid exacerbating the hostility already suffered by people from marginalized groups. I myself routinely offer critical feedback on DEI efforts at CSE/HDSI. Inclusivity and freedom of speech are not mutually exclusive.
Likewise, Big Tech companies are facing heat for exacerbating DEI issues in society. Accountability is crucial. Nations need prudent new laws/regulations to reduce disparate impact of new tech. But I doubt we can move the needle on DEI in CS by shunning Big Tech, the biggest employers of our students. I hope CS industry can introspect about their new power and earnestly partner with governments, nonprofits, and academia as allies in DEI efforts. We are all in this together. Looking further out, most Millennials and Gen Z are sounding the death knell to runaway Capitalism’s era of “profits before people/planet.” I hope Big Tech uses their power to help Capitalism itself evolve for the better this century. Their efforts on combating human-caused climate change (e.g., this, this, and this) are a good start.
Finally, to new Asst Profs from marginalized groups, I’d offer two tips. First, form a support network of faculty (on campus and beyond), staff, and industry friends to lend moral support to each other. It is easy to feel isolated and even tokenized in academia due to its small size (vs industry). Going to D&I-focused conferences such as Grace Hopper, Tapia, or oSTEM can help. You are not alone. Second, there will sadly always be people who are deliberately cruel. Ignore such malicious voices, focus on your own voice, and power ahead. I have found that my meditation practice helps me on this front.
Lesson: Welcome constructive criticism of DEI efforts to improve them. Partner with CS industry to amplify DEI impact. Also: Haters gonna hate, hate, hate. Heartbreakers gonna break, break, break. Shake it off. Shake it off. :)
Personal Life and Health
One of the saddest aspects of being an Asst Prof in CS these days is the high stress it causes to both mental/physical health and relationships. I have heard of issues ranging from burnout to postponing family plans to divorcing their spouse. :( I suspect some of these stem from a “cult of overwork” that seems endemic in both CS and academia. Of course, productivity metrics do matter to an extent, but how to achieve a better balance as Asst Prof?
My answer may be surprising: do not “care” about tenure! :) Let me explain. I have written before about my coming out journey. A powerful psychological tool I intuited then thanks to my meditation practice is something I call pre-emptive managed trauma (PMT). It is like a vaccine for the mind. If I want X, I first make peace with not having X. I imagine a future in which I am okay even without X. Note X can be anything: an object, a job, or even a person! I grieve for a while and then move on. This reduces fear of loss/failure. While it may sound odd, the result is positive: if I do get X, I feel gratitude, not entitlement, because my imagined future did not pan out! I suspect this tool is rooted in the fact that we all mostly “make up” our happiness.
Anyway, PMT has helped me with many big life decisions: coming out to my family, during dating, and during job search. Naturally, I applied it to my Asst Prof job too. To be fair, I had many academic job offers in 2016; so, I knew I’d likely be able to move to another good school if UCSD said no. And in the “worst” case, I knew I’d likely be able to find a good job in CS industry — and get paid 2x-6x more! :) Hmmm. :-/ Anyway, we are lucky in CS to have such “backup” luxury, unlike, say, the humanities or even physics.
Applying PMT to tenure liberated me to live a lifestyle with a mix of choices I deemed prudent. Of course, if Asst Prof duties do not fit well in your mix, this job may not be for you. My husband and I love to travel and take “vacations” combined with conferences or in between quarters. It helps that top DB conferences are often held in touristy cities worldwide. I also like solo trips to go hiking in nature. Outside California, I like hiking in Arizona, Washington, Oregon, Wisconsin, and upstate New York. During a vacation, I avoid emails and delete work-related apps and Twitter on my phone.
San Diego is also a beautiful outdoorsy city with lovely weather all year. I love hiking at Torrey Pines. Sometimes I chill out on La Jolla’s beaches. I typically review papers/proposals sitting at Windansea Beach, sipping a cold drink. :) I enjoy watching many genres of movies, both in theaters and on streaming platforms. I am a huge MCU buff. I also love X-Men, DCEU, and some other franchises. San Diego is the world capital of comic culture! So, I went to Comic-Con once, as Thanos. :) Why am I telling you all this? Not to “brag” about my lifestyle choices but to make a key point: happy lifestyle choices help raise productivity! Such fun activities, vacations, etc. likely made me more focused/productive during work weeks. It is a virtuous cycle.
Lesson: Make peace with not getting tenure and even not staying in academia. Figure out the lifestyle you want for yourself. Work backward from that to fulfill your mix of Asst Prof duties. A happy lifestyle can raise productivity.
All that said, I was still stressed out in many situations. As a grad student, this was mainly due to paper rejections; as Asst Prof, due to proposal rejections. The most painful was my first CAREER rejection. It did make me ponder if I even belonged in academia. In hindsight, I had failed to apply the PMT tool I explained above. I also realized I was naive in how I dealt with NSF, e.g., I did not speak with the program officer beforehand to seek advice. It turned out my proposal was misrouted to a data mining area panel instead of a database area panel, leading to strange reviews/expectations. :-/ The second rejection was much less painful, in part because I had applied PMT by then, but also because I also had other sources of funding.
Of course, there are many other factors that helped me achieve some balance. First, I am fortunate to work with excellent students at UCSD who are diligent, sharp, and self-motivated. I did not need to do much handholding. I think my advising practices also helped on this front — see Part 1 on advising. I set reasonable communication boundaries, e.g., they know I will not be available for research updates/questions during my vacations. So, they plan their papers, revisions, etc. accordingly. Likewise, I always advice them to take enough breaks. But many ignore that advice often. Hmm, is there a cynical “reverse psychology” tool for advisors here? ;)
Second, I am thankful to my husband for all his support. We did face many occasions of marital stress (does any couple not?). But we always pulled through with some help, including from close family and friends. Moving to a new city was a key factor for his stress; for me, it was my work schedule. I’d often work from home on weeknights and even some weekends near paper/proposal deadlines and for quiz/exam grading. This affected my sleep schedule and likely made me crankier. But I always get 8–9 hours of sleep. I try not to let work pile up till deadlines, but it is not always under my control alone. For my next stage, I plan to avoid emails most nights and weekends and put my foot down on my boundaries with collaborators.
Finally, we do not (yet) have kids, which I am sure enabled me to focus more on my work and gave us more mobility. But I do know many CS Asst Profs with kids. I do not know how they manage it all. My husband also took up way more than half of our household chores. To be fair, he enjoys that and refuses to offload some tasks to me because he says he takes greater care with the details of laundry, cleaning, etc. than I do. :) As a gay couple, we are also free of the (often unconscious) fetters of patriarchal heteronormative sociocultural conditioning that affects many straight couples.
Lesson: Your loved ones will matter more than you think. Communicate with your spouse/SO honestly about stresses. Seek help from close family and friends. Being vulnerable is a strength. And hang in there!
Other Overall Observations and Looking to the Future
Finally, a miscellaneous list of things I wish I knew earlier, mistakes in hindsight, and things I am still uncertain of as I look toward the future.
1) I wish I knew sooner about the pitfalls of publishing intersectional work. It was tedious to re-pitch new problems in the DB+ML+systems intersection to different audiences when I shopped some papers across areas. I am not sure if all that hassle was worth it. In the end, good ol’ SIGMOD/VLDB ended up sufficient for all my full research papers.
2) A flipside of the above is that my work’s visibility is poor in the core ML/AI world. Then again, I never attended any top ML/AI venues because my work is not on new ML/AI algorithmics. It still surprises me that this was not an issue for any of the dozens of senior folks who read my file! Yet, I still feel I could have done / should do more on this front. Maybe a tutorial at such venues on data management/systems issues in ML/AI? Will it matter for something? I do not know. But I am glad to see “data-centric AI” and “MLOps” work gaining more visibility at such venues now.
3) In Part 1, I explained the risks/pitfalls of the “one big system” approach to academic research. Yet, I do wonder sometimes how that might have gone. There are many benefits to such an approach: it focuses attention and funding and offers more bandwidth for translation to practical impact via open source and/or startups, assuming the project succeeds. But is the dichotomy really so stark? For instance, in the last 2 years, my attention/funding/efforts are concentrating around 2 “big” but complementary projects: Cerebro and SortingHat. Ask me again in 5 years about how it goes.
4) My colleagues/mentors told me early on to serve as an NSF panelist ASAP. I failed to take it seriously. That was a key reason for the debacle with my first CAREER and in fact, 3 more early proposals (Small, Medium, and CRII)! I was too shy to offer to be a reviewer because, well, I had never written a proposal and wanted to see reviews in my area first. In hindsight, I made a false equivalence with paper reviewing. I’d advise all Asst Profs to beware this mistake. Screw being shy! It is better to be a shoddy reviewer than to receive shoddy reviews. :)
5) I still wonder if I should have pursued more military research funding, a huge avenue for CS research in the US. DARPA in particular was behind many CS breakthroughs. I did peruse the proposal calls of DARPA, ONR, AFOSR, and ARO early on. But they list specific topics, and none were particularly relevant for me. The only relevant DARPA program I found was ending. So, I just left it at that. Still, I’d advise all Asst Profs to study the calls of such agencies, including their “young faculty awards.”
6) Some of your students will disappoint you once in a while despite your best efforts. You may not be able to make it better either. If it recurs, part ways graciously. Some may switch advisors themselves. Either way, I always put the student’s long-term interest first. Such issues can creep up on you or happen suddenly. I wish I knew how to predict them sooner. They hurt for a bit, but we all move on soon enough.
7) It is natural to feel awe and/or envy for peers in your area and age group whose accomplishments you admire. To me, this included Peter Bailis, Tianqi Chen, Aditya Parameswaran, Andy Pavlo, and Matei Zaharia. While awe is helpful, envy can be self-destructive. Thankfully, meditation helped me transform envy to mudita over time. Competition is inevitable, but it is unfair to yourself for you to keep comparing with folks with different circumstances (visible and invisible). Also, research is not a zero-sum game. We can all learn from others’ paths, be inspired, inspire, uplift, and bring our original tones to the eternal grand orchestra of scientific/technical progress.
8) We are in the golden age of CS Asst Profs launching research-based startups. California is awash with venture money in the areas of MLOps, data-centric AI, ML/AI applications, data engineering, and cloud-native systems. All of those are up my alley. Many of my ex-advisors, colleagues, and friends have also done successful startups. No wonder I am asked often if /when I will do a startup. Some faculty also do stints at an established company instead to help innovate on products and/or learn about real-world challenges. None of these avenues were priorities for me as Asst Prof. But as said in Part 1, I did help transition some of my research to products by collaborating with some key companies. For my next stage, I am now reevaluating all the above avenues. Ask me again in 5 years about how it goes.
9) Finally, my husband and I are excited to welcome a new member of our family soon: a German Shepherd puppy! We know it will be a lot of work. But after I got tenure, he quit his banking job, in part to ensure we have enough flexibility to care for the puppy and time for ourselves. This is a privilege that not all couples may have, but we consciously worked out a balance that works for both of us. I hope it all goes well.
Concluding Remarks
Even as little as 7 years ago, I could not have imagined where I am now in life. So many big changes: came out of the closet, dating, got engaged, got a PhD, got this job at UCSD, got married, bought a house, US permanent residency, and now tenure. I am thankful to the many people in my life — both personal and professional — who helped me along the way. I hope this memoir about my experiences, strategies, tools, pitfalls, successes, debacles, joys, and pains as Asst Prof is helpful to folks pursuing or considering this path. Of course, everyone’s journey is unique. Adapt my lessons to yours. I also hope it helps demystify and humanize to the world the lives of a new generation of CS Asst Profs. Thank you for reading. Best wishes!
