generative glitter #03
Domain knowledge, accounting fail, workplace strategy, brainstorming buddy
generative glitter is a weekly newsletter focused on how to use generative AI to work better—from theory to techniques. Let’s go!
To me, the process was a bit akin to hip-hop [...] you don’t necessarily know how to drum, but you definitely need to know how beats work, how hooks work, and you need to be able to put them together in a meaningful way.—Stephen Marche on writing "Death of an Author"
First up in the news, Nextdoor is continuing their self-transformation to promote more welcoming neighborhoods by creating an AI assistant that helps neighbors rewrite posts to be more kind.
This is a good example of how specific domain knowledge can be used to engineer prompts that are targeted for a specific use case. Here, Nextdoor is using ChatGPT with some custom prompting to make a social network less toxic.
Meanwhile, Chegg, an edtech company that helps students study, lost 42% of their share value this past week after their executive hinted that ChatGPT was responsible for slowed customer growth.
The Washington Post covers it neatly and suggests that this might be an overreaction. They note that Chegg’s competitive advantage is not in the base model—the GPT that knows how to string words together—but rather in the training data—the “decades of experience and a growing corpus of proprietary educational materials developed by humans.”
And, as you might guess, Chegg is already working on integrating GPT into their automated tutor.
Some evidence
Researchers from 186 universities around the world found that GPT-3.5, the model behind the public ChatGPT, wasn’t as good at answering accounting questions as an average student. They put it to the test with over 25,000 textbook and exam questions and found that overall students scored 77% while GPT scored 57%.
The model struggled more with short answer and workout questions and with financial, managerial, and tax topics (i.e., the tricky accounting stuff).
Of course, the AI world has changed dramatically since the study was conducted in January. Base ChatGPT itself seems to have improved on quantitative tasks and GPT-4 is reportedly even better. But, when I tried this accounting-ish example from Chris Hoffman:
I currently have 4000 apples and 2500 oranges. I’m buying more fruit and I want to have a 50/50 ratio of apples to oranges. I’m going to buy 4500 new pieces of fruit. How many apples should I buy and how many oranges should I buy?
ChatGPT got it right 6/10 times. ❌🍏❌🍏🍏❌❌🍏🍏🍏
This is maybe fine, since GPT is really a language model (not a calculator) and someone who has domain experience could catch it (in this case, I’m the fruit expert).
But, if you used GPT to do your taxes this year, you might want to double check them… which ultimately supports the idea of still having a human in the loop.
Meet your new coworker
On Inc.com, Sarah Lynch writes about tactfully introducing employees to AI in the workplace… because, since people still do work, it is still important to consider their perception and experience. She outlines a strategy:
Demonstrate examples
Get employee feedback
State risks and guidelines
Get more employee feedback
From a business perspective, the risks of inaccurate AI output could be a barrier to entry for many organizations right now. As an alternative approach, Elaine Burke writes for The Hill about how a collaborative “human in the loop” model could empower workers, echoing research that I mentioned last week about AI assistants.
Yet, the way businesses envision the future of AI in their workplace has major implications for this strategy (and for the employees). For example, if a business is simply biding their time until AI outputs are good enough to be used without a human intervening, then positioning these tools as a “personal assistant” would be disingenuous.
Indeed, the New Yorker this week questions the possibility that generative AI can be used for anything other than as “capital’s willing executioner.” While an alternative model (the “augmentation” path) certainly exists, implementing it effectively will require transparency and guidelines.
As an example of a company getting ahead of this with thoughtful policies, WIRED has set clear boundaries for their employees that they cannot use unedited text generated from GPT, but they can use it for tasks like brainstorming or editing.
I have a feeling this topic will gain prominence soon as more organizations flirt with introducing AI into their workflows.
So let’s break down how this might work for a business.
First, talk to your employees about their pain points at work and identify parts of the job that could be enhanced with AI, like tasks that are repetitive or require a shared voice.
Second, decide if you want to augment or automate those tasks.
If augmenting, think about the potential use cases: brainstorming, continuing education, training, editing, ...
Then, give examples and show employees the happy path of how you would like them to use the tools to achieve the task.
If automating, make it clear what tasks will be automated and what will replace that space for an employee (if there is nothing to replace the space, then you might really be on the “willing executioner” path that the New Yorker outlines).
Then, set up a transition plan to help current employees leapfrog to a re-framed job description.
Third, measure, learn, repeat!
What do you think about workplace AI policies? How should companies approach this?
Expand Your Mind
Inc.com is on a tear right now with Leading Tech articles about generative AI. In another one of their articles this week, they summarize an article written in WIRED that highlights the power of generative AI for brainstorming. Here is my summary of the summary. They break it down into three steps:
Identify the core question
Be persistent with prompts
Ask for lists
Both the original and the summary are worth a read.
So, let’s give it a shot to get from zero to one. I’m going to pretend to be in marketing and try to plan a new campaign. I tried this:
I'm making a new marketing campaign for my running shoe company. Our core market is trail runners who like the outdoors. We are advertising on Instagram. Give me 10 ideas for creative marketing campaigns.
This gave me some awesome ideas (to me, who is not a marketer and does not own a shoe company). The one I liked the most was:
"Shoe Selfie Saturday": Encourage followers to share their trail running shoe selfies every Saturday, using a branded hashtag. Feature the most creative and captivating photos on your Instagram page, creating a sense of community among trail runners.
So I said:
Great, give me more detail for #5, including hashtag ideas and how to encourage people to join.
And I got back some more helpful guidance. First up:
Make a clear announcement about "Shoe Selfie Saturday" in an Instagram post or story. Explain the concept, share examples of shoe selfies, and mention the branded hashtag to use.
Sounds good to me. #ShoeSelfieSaturday
—Aaron
Want to explore how to apply AI in your work? Reply to set up a free discovery call with me. I want to hear about your ideas!