generative glitter #04
Check your work, basic results, thick personality, roll-out requirements
generative glitter is a weekly newsletter focused on how to use generative AI to work better—from theory to techniques. Let’s go!
“AI may not replace managers, but the managers that use AI will replace the managers that do not.”—Rob Thomas, chief commercial officer at IBM, as reported by TechCrunch
First up, I’m excited to announce the launch of our first mini-course! 🚀🚀🚀
» Sign up for the FREE generative glitter workflow course « to start experimenting!
In the news, Slack announced their integration with GPT last week. Slack is a major application for workplace communication, so the introduction of generative AI features has the potential to give a wide range of users access to these tools where they already work (as opposed to the long tail of “thin-layer” applications that need to introduce new workflows for users). The announcement showcases summaries and edits.
Based on this week’s content, it’ll be interesting to see how well they do out of the box.
Some evidence
You gotta check your work.
This week hCaptcha, a human verification software company, reported an interesting (albeit cringe-worthy) experiment where they posted a request on Upwork (a freelancing platform) for a SQL analysis. The twist: the question was designed to trick large language models despite being correctly answerable by a domain expert.
They found that 9 (out of 14) bidders answered the question. And all of those answers fell for the trap (answering incorrectly), suggesting that all of the responders used generative AI for their bid without reviewing it. And a few used generative AI for their reply message too.
The lesson? As a user of generative AI, you gotta edit your work. And hCaptcha suggests that, as a consumer of content that could be generated by AI, you gotta review the work that you receive.
Basic results
The New York Times loves posting exposés about AI these days. This week they published two, one with a case study about channeling all workplace communication through GPT for a week and the other about letting GPT choose what clothing to wear to work.
The output in both cases was stereotypical and lacked personality.
For example, the clothing assessment recommended that the author, Emma Grillo (a “28-year-old woman who works at a news organization”), should wear “tailored black dress pants, a white button-up blouse, a black blazer and a pair of black closed-toe pumps” (very different from what she was actually wearing).
The workplace communication experiment was just as bad—after a message exchange, the author’s friend noted that the AI was “very polite, but in a way I’m slightly concerned might hide her intention to murder me in my sleep.”
The authors raise important questions about the value of individuality and the boundaries of outsourcing relationships. The dystopian view is a world where we all listen to the same advice and wear the same clothes and our AI bots talk to each other in the same trite tone.
But, both cases share the same challenge of leaning on a foundation model for very targeted use cases. Out of the gate, the results kind of sucked. It wasn’t until Emma gave more specific details (like, “dress me in the style of Elizabeth Holmes”) that the results became slightly distinct.
So how does one introduce specific personality into a generalist assistant? You need to teach them.
Let’s use my favorite example tech company, StitchFix. StitchFix doesn’t recommend the same clothes to everyone; they learn what you want. A foundation model is always going to generate basic output, because it doesn’t have any context. No description of my personality. No examples to follow.
For StitchFix, the features that they use to pick clothes are your style configurations and other items that you like or dislike. For a language model, the features are the words you use in your prompt.
Let’s dig in to some examples.
A layer of personality
A few months ago I wrote about the different parts of an AI prompt.
And in a previous issue I gave an example of how to provide context in a prompt with some descriptive language.
The context language and parameters set the model up for success.
Getting a foundation model to have a specific style (or personality) takes a little bit of effort. But, luckily, large language models are fast learners. Even a few specific examples can teach the system to respond in a way that you like.
Michael Dempsey calls this a thick layer of personality—anything that gives an opinionated voice to an application layer. The cool idea here is that, even if the application is a “thin layer” on top of a foundation model, it can have “thick personality” (the VC crowd loves talking about layers).
Let’s imagine that I get a lot of customer emails that I need to reply to every week. I might want a way to speed that up.
I turned to ChatGPT and tried this:
I am a startup founder who wants to provide excellent customer service. To put service first, we always offer a full refund. Write a short and thoughtful reply email to this customer message.
Customer message:
I emailed the other day to ask about a refund but I didn’t get a reply. I am a florist and it turns out that your accounting software does not have all of the features that I need, especially expense management. Can I get a refund?? - Steve
This was the result:
Dear Steve,
Thank you for reaching out and bringing this matter to our attention. I sincerely apologize for the delay in responding to your previous email; it seems to have slipped through the cracks, and I'm sorry for any inconvenience caused.
As a startup focused on providing excellent customer service, we deeply value your feedback and want to ensure your complete satisfaction. I understand that our accounting software does not meet your specific needs, particularly regarding expense management, which is essential for your florist business. Please accept my sincerest apologies for any frustration or inconvenience this may have caused you.
In line with our commitment to putting service first, I want to assure you that we will gladly provide you with a full refund. Your satisfaction is of utmost importance to us, and we want to make sure you have a positive experience, even if our software wasn't the right fit for your business.
To initiate the refund process, please provide us with the necessary details, such as your purchase information or any other relevant information we might need to proceed efficiently. Once we receive the required information, we will promptly process your refund, and you should see the amount credited back to your account within [timeframe].
Again, I apologize for any inconvenience caused and assure you that we take your feedback seriously. We appreciate your understanding and hope to have the opportunity to serve you better in the future. If there is anything else we can assist you with, please don't hesitate to let us know.
Warm regards,
[Your Name]
[Your Title/Position]
[Company Name]
Yikes. Even with some specific context, it sounds nothing like me.
Now, let’s try something a little bit different with some examples, called few-shot learning:
I am a startup founder who wants to provide excellent customer service. To put service first, we always offer a full refund. Write a short and thoughtful reply email to this customer message.
#Customer message:
I don’t understand why your accounting software doesn’t have expense management built in. As a baker I need that functionality! It makes your software unusable. I need a refund.
Jan
#My reply:
Hi Jan,
Thank you for reaching out. I know that our functionality is currently limited and I understand your perspective.
I am refunding your subscription right now. You will still have access to the platform through the end of the year.
We have expense management on our roadmap and we hope to have that added before the end of the year. I hope that we can keep you as a customer!
Sincerely,
Aaron
#Customer message:
Your software is good but it doesn’t do everything that I need. I would really like an invoicing feature. I’m switching back to QuickBooks until you’ve got that. But, I signed up for a whole year. Can I get a refund for the remaining months?
Ricky
#My reply:
Hi Ricky,
Thank you for reaching out. I understand your perspective and I’m sorry that we don’t have everything you need right now.
I’m giving you a refund for the remainder of your subscription. Can I keep you on our newsletter so that you can stay up to date for when we add invoicing to the product?
Sincerely,
Aaron
#Customer message:
I emailed the other day to ask about a refund but I didn’t get a reply. I am a florist and it turns out that your accounting software does not have all of the features that I need, especially expense management. Can I get a refund?? - Steve
#My reply:
And the result:
Hi Steve,
I apologize for not getting back to you sooner. Thank you for letting me know about the issues you're facing with our software.
I have processed a full refund for your subscription and you will receive the funds in your account shortly.
We're continually working on improving our software, and we appreciate your feedback. We'll definitely take your feedback into account as we make further improvements.
If you have any other suggestions or feedback, please don't hesitate to reach out.
Sincerely,
Aaron
Better! It’s starting to match my style a little bit. I could revise that in a minute and ship it (probably faster than writing from scratch).
Now, imagine having like 5 (or 50, or 500) messages for tuning. In no time, you can have a tool that writes with your voice for all kinds of cases—a thick layer of personality.
Roll out strategy
Kevin Delaney writes in The New York Times (I swear I read other sources…) with more thoughtful advice about rolling out AI in the workplace. They highlight the opportunity to augment employee expertise with an example from Morgan Stanley; an initiative to build an internal generative AI knowledge-base. Imagine a system where employees can ask natural language questions about the business, the customers (see also Salesforce’s Einstein), and beyond.
For companies who want to introduce AI to their employees, Kevin identifies a few helpful requirements:
A base of data to leverage for business-specific insights
A technical expert to design the system
Domain specialists to tune the content that is generated
Most companies have some data. The technical expert could be a third-party consultant (you can hire me) or a new employee (for a large company like Morgan Stanley). And guess who the specialists are? They already work for you. They’re the early adopters who are experimenting with tools like ChatGPT to do their job right now.
This makes experimenting with generative AI seem feasible for businesses of almost any size.
—Aaron
Want to explore how to apply AI in your work? Reply to set up a free discovery call with me. I want to hear about your ideas!