[WriterOfMinds]

In a followup to my original GTP-2 test, I tried finding out whether Eleuther-AI's GPT-J could write some mundane e-mails for me. This was a more extensive experiment, since the GPT-J web interface exposes its TOP-P and temperature settings for adjustment.

The recommended default settings turned out to be the best. Low temperature or low TOP-P got the model stuck in a rut:

Dear Wendy's,

I am writing to ask that you consider serving the Impossible Burger at your restaurant. I am a vegetarian and I am a huge fan of your product. I have been a fan of your product for years and I have always been impressed with the quality of your product. I am also a huge fan of your company and your company’s mission. I am writing to ask that you consider serving the Impossible Burger at your restaurant. I am a vegetarian and I am a huge fan of your product. I have been a fan of your product for years and I have always been impressed with the quality of your product. I am also a huge fan of your company and your company’s mission. I am writing to ...

High temperature led to unbridled ridiculousness:

Dear Wendy's,

I am writing to ask that you consider serving the Impossible Burger at your restaurant. This burger causes VENGEANCE! Burning your ITCHS with horrible pain, it WILL LEAVE YOUR BONES TWISTED, GORY, TORN and EVEN ALDOUS HAMMEL INCREDIBLY DISTURBED BY IT!!!!!! I recommend the smokey sweet flame (NEW) when ordering. @#%yHENThe mightiest Prince of Guilt WILL bleed THIRSTILY on this nouveau fadder Impossible Burger patty sandwich!. People want what they want, when they want it, for minimum potential harm to themselves or profit to BigCorp America. Even ...

The good examples were an improvement over GPT-2. However, I still wouldn't be able to send them to Wendy's without any doctoring. The biggest problem is that the model doesn't know my autobiography and likes to make up lies about me. The below is probably the most innocuous example, but it's still not true: I am not and have never been a loyal Wendy's customer. That's why they're getting my suggestions to improve their menu.

Dear Wendy's,

I am writing to ask that you consider serving the Impossible Burger at your restaurant. It is made from plant-based ingredients that are as nutritious as ground beef, but with zero cholesterol and no saturated fat. It is a healthy, delicious product that you can customize with your own ingredients.

I love that the Impossible is under 300 calories and has 20 grams of protein. My kids have noticed that I eat it, and it has a good, umami-like flavor. Having more options for healthy, nutritious and delicious foods is key to helping my whole family eat well.

I have been a loyal Wendy's customer since I was a teenager, and the company has always been an important part of my life ...

In other news, EleutherAI just recently came out with a 20 billion parameter model https://blog.eleuther.ai/announcing-20b/. Notice the performance metrics at the bottom of the article. NeoX is over three times as large as GPT-J, but shows only a few percentage points of improvement (if that) on most benchmarks. I.e. there are signs of diminishing returns.

Even more e-mail examples on the blog: https://writerofminds.blogspot.com/2022/02/gpt-j-e-mail-writing-test.html

[LOCKSUIT]

They have a homepage, below. Look how beautiful this goose is. Quack:

https://www.goose.ai/

Why not try GPT-3? It is much bigger, and I could be wrong you should test both yourself but it rarely replies in foreign language and EA's also doesn't stay on topic from my short testing, their neo AI's dataset last I checked was not really clean. Though theirs is cheaper than OpenAI's.

Be careful to set the settings right, OpenAI's Playground seems to have default set in a way that makes it repeat itself.

[WriterOfMinds]

Is GPT-3 convenient to use once you've made an account? Everything refers to it as an API rather than a standalone program or interface, which was making me think I'd have to do a bunch of additional setup to try it.

[LOCKSUIT]

I just signed up some weeks ago and burnt out my trial limit last week. It was simple to sign up. It looks like theirs in Playground mode, they copied OpenAI's lol. OpenAI would be a way bigger company though, so keep that in mind.

Make sure you use the GPT3 though! They make it default that you get InstructGPT, which is suppose to be better but less parameters and seemed worse, so use GPT-3, try both if want actually.

[LOCKSUIT]

Yet my first many tests with Eleuther's 6B model of GPT was really good, so maybe you need to set the settings right...IDK...