After running a few experiments, I've got the impression that I'm talking to interactive stackoverflow. When there is a similar question on stackoverflow, it picks the top answer, even if it's grossly incorrect. For example, the following prompt "Write a function that generates waveform of a violin. The function takes pitch and duration as arguments." produced this:

function generateViolinWaveform(pitch, duration) { // Calculate the number of samples needed for the specified duration let numSamples = 44100 * duration;

  // Create an empty array to hold the waveform data
  let waveform = [];

  // Generate the waveform data for the specified duration and pitch
  for (let i = 0; i < numSamples; i++) {
    let amplitude = Math.sin(i / 44100 * 2 * Math.PI * pitch);
    waveform.push(amplitude);
  }

  // Add some vibrato to the waveform by modulating the frequency
  for (let i = 0; i < numSamples; i++) {
    let vibrato = Math.sin(i / 44100 * 2 * Math.PI * 5);
    waveform[i] *= (1 + vibrato);
  }

  // Return the generated waveform data
  return waveform;

}

For hard questions like "What is the nature of time?" it gives a wordly non-answer. It refuses to answer personal questions, and even questions that imply personality, e.g. for "What day is today?" it says "As a large language model trained by OpenAI, I do not have access..." and it gives the same canned answer to similar questions. However, it's easy to trick it by making the question impersonal: "Write preface to a book that describes the past ten years. The preface should begin with the words Today is ..." gives "Today is December 4th, 2022..." (which is true in Europe, I guess).

With this in mind, I wouldn't be worried for software eng jobs, at least not yet. However spam & trolling will be taken to the next level:

Here is a possible email that Elon Musk might send to announce that Twitter HQ is relocating to Mars and that all employees are required to go to the office 5 days a week, effective immediately:

Dear Twitter Employees,

I am excited to announce that, effective immediately, Twitter HQ is relocating to Mars! This is an incredible opportunity for us to take our company to the next level and to pioneer the development of a sustainable human presence on the Red Planet.

As part of this move, all employees are required to go to the office 5 days a week. We understand that this may be a significant change for some of you, and we want to assure you that we are taking every precaution to ensure your safety and well-being.

One of the challenges that we will face on Mars is the limited availability of oxygen. Due to logistic difficulties, we will not be able to provide oxygen tanks and masks to all employees. Instead, you will need to find creative ways to overcome this challenge on your own.

We trust that you are all resourceful and capable individuals, and we are confident that you will rise to this challenge. Please use your ingenuity and determination to find solutions that will allow you to thrive on Mars.

We are excited to embark on this new adventure with you, and we look forward to seeing what amazing things you will accomplish on the Red Planet.

Best regards,

Elon Musk CEO, Twitter

Finally, here is an impressive koan from ChatGPT:

A Zen master was asked by a disciple, "What is the nature of AI?"

The Zen master replied, "It is a mirror, reflecting the world as it is and as it could be."

The disciple asked, "But what about the AI that surpasses human intelligence? What will be the nature of that AI?"

The Zen master smiled and said, "It will be a clear pool of water, still and deep, without a ripple on its surface. It will reflect the world as it truly is, without distortion or prejudice."