generative glitter is a weekly newsletter focused on how to use generative AI to work better—from theory to techniques. This week we dig into how I might automate myself out of a job and what that could mean for my career. Let’s go!
First up, Meta had a big release last week with Llama 2 coming out. This model is unique because it is open-source and has a commercial-friendly license.
As the MIT Technology Review reports, Meta hopes that releasing it widely and permissively will help them learn how to make models safer and less biased faster and also compete with other organizations like OpenAI.
Already building on top of the Llama models, Stability AI also released two new FreeWilly LLM models that extend the foundation models from Meta and inch closer to competing with ChatGPT (which is kinda the standard benchmark these days).
Both Llama 2 and FreeWilly are available through HuggingFace.
All of this comes hot on the tails of the announcement last week that seven major AI companies (including Meta and OpenAI) have agreed to safety standards proposed by the White House.
Their focus:
✨ Security testing
✨ Generation watermarks
✨ Limitations reporting
✨ "AI for social good"
✨ Bias research
I’m interested to see where this goes, especially the AI for social good—imagining there will be funding opportunities for aligned work in the near future.
Code interpreter
This week I started experimenting with the ChatGPT Code Interpreter and I am blown away. 🤯
As a data scientist in my day job, I do a lot of exploratory data analysis and modeling. The Code Interpreter module can do this. It can accept and analyze raw data uploads— which are deleted after the session is closed. It can take Python code generated by ChatGPT and run it in a virtual environment. And it can save exported results.
Those are basically the core tasks that a data scientist does: import, tidy, transform, visualize, model, communicate. So, I was super excited to give it a whirl.
I decided to experiment with some of my running data from Strava, since I am also training for some big races this summer. Maybe I can improve my training…
Here’s how it went. (Apologies in advance, there is a lot here.)
First up, I had ChatGPT load and explore the data.
…
…
Cool… it corrected its assumptions about the data.
Then, I had ChatGPT analyze my running data as a running coach.
Here's the scatter plot showing the relationship between running distance and average speed, with the size of each point related to the elevation gain during that run.
Each point represents a run, with the x-coordinate being the distance of the run, the y-coordinate being the average speed of the run, and the size of the point representing the elevation gain during the run.
From the plot, we can see that for longer distances and runs with more elevation gain (larger points), the average speed tends to be lower. This is consistent with our earlier analysis.
[We made a weighted linear regression model to predict this with R^2 = 0.467]
The plot shows how your total running distance changes from week to week. There seems to be a fair amount of variation in your weekly distances, which is not necessarily a bad thing. It's common for runners to have "down" weeks every few weeks to allow for recovery.
However, as you get closer to your ultramarathon, you may want to ensure a consistent increase in your weekly distance, while still including recovery weeks. Remember to listen to your body and not push too hard on these long, hilly runs. It's important to recover properly after them to avoid injury.
[We calculated the average rate of increase, which is well below the recommended max.]
From the plot, we can still observe the negative trend between the previous day's distance and the day-to-day variation in running distance. This indicates that, generally, if you run a longer distance one day, you tend to run a shorter distance the next day, and vice versa.
This pattern aligns with a common training strategy of alternating between hard and easy days. After a day of longer distance, it's common to have a shorter, recovery run the next day. Remember that rest and recovery are crucial parts of training, allowing your body to adapt and become stronger. Always listen to your body and adjust your training accordingly.
[We did a correlation to show the negative relationship, r = -0.73. Turns out I need more back-to-back long days, which the model recommends below.]
Amazing… it predicted and interpreted results.
Finally, I had ChatGPT export a PDF coaching report for me to download.
Insane… it gave me recommendations that I have also gotten from my ultra-running friends.
My takeaway
As a data scientist who knows how to ask data-driven questions and think critically about the analysis of another entity (in this case, ChatGPT), I can use this tool as a hands-on assistant.
Pairing the Code Interpreter with the new Custom Instructions module—which saves the context you want the model to know about you and your desired output—creates a serious work augmentation toolset.
It basically enables no-code data science.
What about for folks who are not data professionals?
A colleague of mine who is data-savvy but not an experienced data scientist was also experimenting this week for self-service data analysis. I offered a few suggestions on how to guide their work (“the dotted line is the average, you should have it redraw and add a trend line”), to which they said “I still need you, you’re irreplaceable” ❤️.
I mention this to highlight that having an experienced operator is still helpful but, critically, this experience is NOT TECHNICAL.
For most data analysis tasks (and, by extension, computer programming and writing tasks), this leads us to a future where we need to teach less about the mechanics of writing code or sentences and more about the critical thinking skills necessary to guide the LLM.
Worker context
Last week, Entrepreneur republished an article by Emily Hanley, a freelance copywriter and comedian, about her experience losing work to ChatGPT. It is a great first-person account, including a story about an organization that almost hired her on a term basis to train an AI to do the job longer-term.
Related, earlier this month, Rest of the World published interviews with four outsourced freelancers around the world to talk about how they are using generative AI. They also compare original human-generated output from each with output assisted by a model, showing how the models can increase speed and reduce cost.
The Rest of the World article emphasizes that many of these tasks still require a human operator, but that the focus for some tasks is shifting to “micro tasks”—guiding and training a fine-tuned AI system to do the larger task at hand (imagine the Captcha security prompts).
Ultimately, these examples highlight tasks that could be outsourced to contract workers or interns is starting to be automated.
To think about the potential consequences of this, we can look at an example from the past.
Dylan Matthews writes in Vox about the experience of telephone operators, a profession that was largely automated starting in the 1920s.
What interested me the most in this article was the worker impacts. The existing workers who were automated out of a job had some negative career outcomes (some left the workforce and some took lower-paying jobs). But, the new workers who were entering the workforce during automation saw limited impacts—they simply took different jobs (of which there were coincidentally more available).
The way I think about this, in relation to the Code Interpreter experiment above, is that we need to help workers learn to direct these models. We need bridges for existing workers in disrupted jobs and we need ladders for new workers to learn these (less technical) skills.
Maybe I’m just biding some time (years? decades?) in thinking that there is something special about critical thinking, problem formulation, and communication skills. These could definitely be within the purview of LLMs.
But for now, I think the way ahead is to learn how to guide a system that can create the content instead of learning how to create the content. That is, learn how to become an AI manager.
—Aaron