GitHub Copilot

To my surprise, GitHub Copilot is an astonishingly good tool for developers. In this post, we will look at some AI history and context, then get into some example interactions of coding with Copilot.

Memory lane

I’m an old-school AI sceptic. In the late 90s, I studied neural networks and even trained them to classify the input to sensors on a robot that we built at university (rule one of the robotics lab was that you weren’t allowed to sleep in the robotics lab). Even back then, the lecturers told us about how AI had been promising to be huge for years, and everyone was still waiting.

They told us about a neural net that had been trained to detect tanks in photos. It happened that, in the training set, all the pictures of tanks were on dark days, and the rest were on bright days. Instead of training the neural net to recognise tanks, they had actually trained it to recognise sunny days and dark days. This story may be apocryphal, but even as a thought experiment, it illustrates a point: a trained AI system is only as good as the data it is trained on. 

Things have moved on a lot since simple neural networks, but that lesson of training remains the same. More recently, the DARPA labs trained an AI system to detect human attackers approaching it. The researchers then challenged a group of Marines to approach undetected. The Marines succeeded by moving in surprising ways for humans. Two of them somersaulted for 300 metres. Another pair approached inside a cardboard box, giggling as they went. See this excerpt from “Four Battlegrounds – Power in the Age of Artificial Intelligence” by Paul Sharre. Clearly, the intelligence on display here was far from generalised. 

And yet, my university robot could robustly recognise colours. The AI version was better than when I tried to program in the parameters of which wavelengths it should expect for which colours. 

Since ChatGPT was unleashed on the world, everything is LLM this or GPT that. So, I thought I should suspend cynicism and check out GitHub Copilot with my team. 

Aidan Harding

Technology Director

Looking for help with Salesforce development?

Get in Touch

All about GitHub Copilot

Copilot claims to be an AI partner while you’re coding. Like pair programming, but with the machine. It’s been around for a while, and is available for both VS Code and IntelliJ IDEA. So Copilot sits comfortably in the IDE for Salesforce developers. It’s reasonably priced: $19/month for business use, so it doesn’t have to save much time each month before it pays for itself.

Once the plugin is installed, it gets right to work, suggesting things to you as you code. You don’t have to provide any prompts. It just looks at what you’re doing and suggests code to fill in. That seamless experience is one of the beauties of working with Copilot. It doesn’t interrupt your flow. The last time I tried an AI coding tool, it interrupted my flow, and I gave up in one day. Copilot is constantly there, ready to help but not interrupt.

It seems to me that the skill of getting good results with Copilot is to write code where your intent is clear. That is the kind of code that we ought to be writing anyway. It’s not a new skill distinct from the coding you’re currently doing. You don’t have to learn to be a prompt engineer. You just have to write good code. When you get it right, your intent and your design appear from Copilot as quickly as you can think. 

The seamlessness also means that it’s not just for beginners. My preconception was that an AI tool might only be helpful for a newbie. They might use the AI tool to explain the syntax or idioms that they don’t know yet. This is no doubt useful. After all, everyone is a newbie once we step out of our familiar domains. But it’s great to see Copilot helping even in a domain that you’re fluent in.

Working with Copilot

What follows is a series of suggestions from my first use of Copilot to illustrate the experience of using it to write Apex code. The grey text is what Copilot is suggesting.

In the course of a project, I created a class called ContactClearStandardDdMaxPaymentBatch. The class implements Batchable<SObject>. Then I used the standard Illuminated Cloud functions to generate stub implementations of the Batchable interface methods (side note: IDEs that can generate stub methods like this are still great!): start(), execute(), and finish(). I wrote my own start method:

When it came to the execute method, I started to use our own library for doing list processing operations in Apex in a functional programming style. The library, Nebula Core, lets you work in much the same way as you might for C# or Javascript by filtering and mapping lists instead of writing for-loops. 

By the time I’d constructed one of our iterators, Copilot had a canny suggestion:

Copilot has figured out my intent from the name of the iterator class and the existing context. Unfortunately, Apex does not have arrow functions. But it’s amazing that Copilot inferred that this is the kind of thing I wanted to do. All I had done was construct a fancy-looking iterator.

With a bit more code provided, Copilot makes a decent guess:

That was a correct completion for this line, using an open source library with a tiny footprint on GitHub. So Copilot was probably just figuring out what a call to this method would look like. In fact, that’s a good model for thinking about what LLMs do. They don’t try to answer the questions you give them, they try to respond with what an answer would look like. That’s why they’re good at writing in a particular tone or idiom, but sometimes not so good at the facts.

For completeness, the finished method is below. You may notice that the field reference now is static rather than going via a string, as Copilot had suggested. Either would have worked, but it’s better to be concrete with fields unless we have to use strings.

For a class where the name does not tell you everything that you need to know about what it does, we include Javadoc-style comments to summarise it. And Copilot was able to generate such a summary perfectly well:

Later, I was working on a class from Nebula Core. This class has a single constructor, and then a number of builder-style functions to set optional parameters. Having added an extra setting to the class (relatedTo), I started to add the builder method corresponding to this setting:

Now Copilot is slowing me down because I’m picking my jaw up from the floor. I wanted to write a new builder method for relatedTo. All I had written so far was “global”, and Copilot wrote the rest of the method exactly as I would have written it. Wow.

One negative experience in my work with Copilot was when the code it produced was hard to check visually. It introduced a subtle bug in the test for the first example above:

Which looks like an amazingly correct usage of our library for generating test records. Except that the correct value for Payment_Method__c should have been ‘Direct Debit Monthly’, not just ‘Direct Debit’.

That slowed down the coding flow as I had to check the entry criteria for the batch class and then notice that subtle difference. It seems reasonable to suggest that these kinds of errors are the dangerous side of using Copilot. Anything that is hard to check is a point of danger. So perhaps it will be better in strongly typed languages where the IDE can identify many errors statically. Or, if you are working in a language like Javascript, then the IDE linters will be even more valuable when you’re using Copilot.

Where next?

When Copilot suggests calling non-existent methods, we could turn that to our advantage. Sometimes those are methods that ought to exist. Copilot is drawing on the traditions set by its training data and the things that it imagines are the sorts of things that human programmers may expect to exist. So, some sort of Test Driven Development approach with Copilot could be a good way to write the interface for your library code.

It’s certainly the case that I’m writing code faster using Copilot. Primarily by letting it work as a beefed up version of the code templates offered by an IDE. An IDE could have generated a builder method like the one above, if it had the right template, and I could choose that template fast enough. But Copilot can act as if all possible templates already exist and suggest them without me having to search. So that’s a really nice way to work.

I do wonder if Copilot will be less effective in spaghetti code or more generally in making updates to an existing code-base. It has good greenfield suggestions, but it may be quieter on modifications. Time will tell on that one.

These are exciting times. It’s the hallmark of a revolutionary product when it offers something that you didn’t know you needed, but it becomes something you would never give up. And that makes me excited about the future of AI in other domains. It will be used in ways that we can’t see now, but will look obvious in retrospect. 

You can read about Salesforce’s AI offerings on our blog in the article Salesforce AI: An Overview.

If you have feedback, comments, or tales of your own experiences with AI then please do get in touch at: @AidanHarding on Twitter, @aidanharding@fosstodon.org on Mastadon, or aidan.harding@nebulaconsulting,co.uk on email.