Why AI Copilot?

Writing code takes time. You often have to write many repetitive code patterns, and writing the same stuff over and over isn't exciting. You also have to remember the correct syntax. Sure, that's not an issue for operations you use regularly, but those may not be enough when you're trying to solve the hardest problems effectively. This becomes even more apparent in data notebooks, where you're frequently dealing with external libraries and tools. Data teams doing exploratory analysis aren't necessarily familiar with the exact structure of their data before they start. This leads to a situation where it's easy to make typos, mistakes, or even critical errors. That’s why we decided to build Deepnote AI Copilot. It’s an integrated AI service that displays predictive inline suggestions (or, as Anton from Ramp likes to call them, ghost text) as you type. We have decided to partner with our friends at Codeium to provide cutting-edge code suggestions for both Python and SQL, making the most of Deepnote’s multi-lingual environment.

Building context

Large language models produce better results when given appropriate context. The same applies to the language model powering Deepnote's AI Copilot. To build a comprehensive context, we've constructed a context module made up of several key components, each contributing in a different way:

Naturally, the main contextual component is the notebook's contents, as they represent the current state of your work. These contents are likely to include your initial steps as well as any updates you've made, giving us a clear picture of your intentions and potential next steps. We can take advantage of the notebook's unique characteristics, like its content being divided into individual cells (or as we like to call them, blocks). This lets us select the most relevant blocks based on your current position in the notebook or the programming language you're working in.

In some cases, your notebook's contents just aren't enough, like when you're starting with an empty notebook. Passing in notebook metadata as context will hint to the language model your primary goal and the best way to get you started. This is enhanced by Deepnote's organizational structure, where you can group your notebooks into projects and categorize those projects into folders, creating an even better context. Notice how naming the project and notebook sets us off to relevant completions:

References to variable names are already part of your notebook's contents, but that doesn't give Copilot the full picture. That's why we supplement this by pulling data from Deepnote's variable explorer, which holds references to all variables created during your machine's runtime. Each variable is passed along with its Python type and additional metadata, like element length in the case of an array. This comes in handy when working with DataFrames, as Copilot knows their column names and data types. Because DataFrames are first-class citizens used to display the outputs of your blocks, you can reference them right away in the rest of your notebook. Take a look at how Copilot picks up a reference to a column name from a text block:

Leveraging existing work across your workspace helps us give you an even better context. We search your workspace for blocks that might be relevant, scoring them based on how often you use them and other factors—like if they're in a regularly scheduled notebook or part of a published app. Having this unique context is particularly useful, not only when you're starting with an empty notebook but also when you're working in SQL blocks. Imagine you need to query a database. Deepnote AI Copilot knows that you are using PostgreSQL and often join the users table with the orders table. So when you begin typing SELECT, Copilot predicts you might be interested in a query like SELECT first_name, last_name FROM users JOIN orders ON users.id = orders.user_id. It understands your SQL habits, the SQL dialect as well as your preferred column-naming conventions.

AI Copilot's context updates with each keystroke, specifically tailored for the block you're currently editing. The block's position in the notebook, its type, and additional metadata are key inputs to the contextual module that determine what information is passed as context. While providing more context generally improves the quality of the model's outputs, too much can be distracting and negatively impact output quality. So, the contextual module aims to not just select relevant data, but also to weed out irrelevant information.

Generating completions

With the right context, we can finally generate some code completions. But context actually isn't the main input for the model. The primary inputs are the content of the block you're editing and where your cursor is in the notebook. For both, we need to calculate position offsets from the start of the notebook and the block, respectively. Getting these offsets right ensures not only relevant code completions for what you're currently typing, but also the accurate placement of generated completions relative to your cursor.

This is especially important for inline fill-in-the-middle completions. Besides completions at the start of an empty line or at the end of a line, Deepnote AI Copilot can generate completions for positions that are sandwiched between existing code, which becomes extremely useful when making edits. No matter the position of the resulting code completion, all completions are generated via a request to Codeium with an encoded payload of the model's inputs. The response we get is decoded similarly. Code completions are first normalized, and then turned into objects containing the completion's text and range. This lets us display them right where your cursor is. Deepnote uses the Monaco editor, which also powers VS Code, allowing us to use Monaco's native inline completions API for showing the generated completions.

Completion post-processing

But if we were to show the generated code completions right away, we'd miss important steps such as filtering out irrelevant ones or keeping your current Deepnote workflows intact.. Large language models can make stuff up sometimes, and Deepnote Copilot is no different. While we haven't seen it invent a new pandas filtering method yet, it can spit out completions that don't really fit. For instance, the if name == "main" completion doesn't make sense in a Jupyter notebook environment like Deepnote. We catch and remove such irrelevant completions. Similarly, we skip over suggestions that are just a single character long (after removing whitespace). We find these one-letter completions are more of a distraction than a help, so we leave that for you to fill in manually 😉. Another feature that speeds up your coding is autocomplete, also known as IntelliSense. This tool offers method and variable name suggestions through a scrollable widget, and it's particularly useful for writing SQL statements since it knows your database schema. However, some early Copilot users told us that the two features can clash, each vying to suggest your next move. To resolve this, we've tweaked Copilot to step back when autocomplete is the better fit—for example, right after you type a period in SQL blocks.

Tips and tricks

Here are some tips to get the best out of AI Copilot, even if you're up against some unique challenges:

• First off, make sure your Deepnote project and notebook have descriptive names. This will help you get relevant code suggestions right from the get-go
• Use text blocks to outline what you're aiming to do. This gives better context and, in turn, better code completions
• When you load up tabular data like .csv files, you'll expose the dataset's schema, which helps generate completions with the right column names and data types
• Don't like the completion you see? No worries, just scroll through other options using the command shortcut (Option/Alt+[ and Option/Alt+])

Summary

The user feedback to Deepnote AI Copilot has been overwhelmingly positive. People love how spot-on the suggestions are and how they just fit right into their coding flow. Today, Copilot saves our users from typing over 300,000 characters each week - about as many as the number of 'life hacks' one can find on the internet that nobody actually uses.

We're always tweaking things behind the scenes to make all our Deepnote AI features, including Deepnote AI Copilot, even better. We believe that with AI at its heart, Deepnote will revolutionize how data teams solve the hardest problems, making coding faster, smarter, and a lot more fun. If you haven’t given it a go yet, sign up and take Copilot for a spin!

Fundamentals of effective prompting

Deepnote AI is powered by OpenAI’s latest models. Here are a few general tips on how to improve your prompts for better results.

Be specific and include details

Be as specific as possible when writing your instructions. By including details in your prompt, you minimize the chances of the model filling in too many blanks for you. Stating your higher level goal is as important as pointing out concrete methods or functions that you expect to be used. For example:

⁠- Mention the specific Python package you want to use:

# ❌ Don't do
Train a machine learning model

# ✅ Instead do
Train a logistic regression on the training data with sklearn

⁠- When plotting a chart, describe how you would like it to look like:

# ❌ Don't do
Plot a chart of the data

# ✅ Instead do
Plot total_sales against time on a stacked line chart for every continent

⁠- Specify the length of the desired output:

# ❌ Don't do
Read the CSV file

# ✅ Instead do
Read the first 100 rows from events.csv

Use appropriate structure and formatting

Leverage formatting for a clearer structure and highlighting the key parts of your prompt. For example, you can make use of operators to outline relationships between parts of the prompt or enclose a value in quotation marks to ask for literal values.

If the problem is complex, you can point out the necessary steps as individual bullet points or numbered points if they need to be completed in a certain order. We recommend breaking down your problem into subproblems and tackling these with individual prompts instead of all at once.

Provide examples

One of the most effective prompting techniques is “few-shot” prompting. This entails including relevant examples in your prompts that demonstrate to the AI exactly how it should approach solving your problem. This becomes especially useful when expecting a specific output from Deepnote AI, as it needs to understand the relevant schema beforehand.

Advanced prompt techniques

General tips will only take you so far. Use the following advanced techniques to fully utilize Deepnote AI’s abilities to understand the context of your workspace.

Leveraging notebook contents

Deepnote AI factors in the contents of your notebook in its outputs. You can take advantage of this and write out relevant semantic information or business logic directly into your notebook so that Deepnote AI can follow these instructions. Bonus tip: try placing the most important instructions closer to where Deepnote AI is working to prioritize them within its context. This strategy works especially well in SQL blocks. Notice how Deepnote AI picks up on our search query for companies in the financial sector:

Referencing notebook variables and files

One of the most powerful features of Deepnote AI is the ability to reference your existing variables and files directly in your prompts. Simply mention variables by name and Deepnote AI will pick them up, including their data type. In case of a DataFrame, Deepnote AI also has access to its columns and their respective data types. When referencing a file, Deepnote AI will check existing notebook variables and automatically import it if it’s missing.

Iterative prompting

It’s possible that Deepnote AI doesn’t get things right on its first try. If that’s the case, there are several ways you can iteratively achieve your desired output:

Clicking Undo/Delete blocks will take you a step back, giving you the ability to edit your prompt. This is perhaps the simplest way to make iterative changes.

If you want to start from scratch, you can always trigger Deepnote AI again in the next block. This is also useful when you want Deepnote AI to continue whatever it was working on prior.

In some cases, you may need Deepnote AI to remember its previous outputs. If you accept your previous Deepnote AI outputs, this will help Deepnote AI remember what it has already tried or you can point out yourself exactly what’s wrong with its previous approaches.

Apart from these tips, Deepnote AI itself might ask for more information about your problem. In such a case, it’s valuable to utilize Deepnote AI’s thinking, accept the current generation by clicking Accept/Done and ask Deepnote AI to continue in a follow up prompt.

Combining all Deepnote AI features

Deepnote AI consists of several features, each tailored for a specific use case. To truly make the most out of Deepnote AI, make sure you’re using the strengths of each one. Auto AI shines in getting you started by generating multiple blocks as well as executing them. With a simple prompt, you will have a solid foundation to build from in no time. It’s also great for exploring possible solutions so you can then prompt it with your hypothesis and quickly check whether it’s worth pursuing further. Once you’re happy with exploration, you can use the Edit command for granular editing and polishing of your notebook.

It’s like generating Python, right?

Writing Python is like using a language you know but writing SQL is like using a map to find your way through a landscape—the "map" being the database schema, and the "landscape" being the database itself.

You need to know the names of the tables and columns to get to the data you want. Compared to generating Python, you also need to provide a “map” 🗺️.

Database schemas are big

Database schemas can be big. Think hundreds of thousands of tables 🤯. The full schema might not fit into the context window of current language models. But there’s no need to cram it all onto your “map”. Your “map” just needs to include tables and columns which are used by the SQL query that leads to your data. But you need to decide which ones are those.

Finding relevant tables

Apart from being large, database schemas can also be quite messy. Table names may use various naming conventions and column names may include symbols or numbers which lose their significance when taken out of their specific context. What if there are multiple tables sharing the same name 😕? Finding only the relevant tables and columns for your “map” is challenging.

Who is a “monthly active user”?

But messy names aren't the only reason why navigating the schema is challenging. To do it well, you need the semantic context about what the data represents and the links in between. Just as new team members need orientation 🧑‍🎓, the AI must learn to identify, for example, a 'monthly active user' based on the specific business logic and conventions embedded in the database schema.

There isn’t just one SQL

Unlike many programming languages, SQL has dialects 🗣️, similar to how English has variants, for example. While most of the syntax and functions stay the same, most databases and data warehouse have their own variant. SQL code written for Snowflake might not work against a Postgres database, because some of the syntax is different. Your solution needs to be fluent in every dialect.

Putting it together

In order to generate a valid SQL which answers the user’s question, you have to find the relevant parts of the data landscape and draw them together on a map. To do that, you need to be a skilled cartographer of data maps, an excellent navigator through the data landscape and a SQL polyglot 💪. Oh, and you want to do it all pretty quickly! 🏃‍♂️💨