To maximise the potential of our engine we need to get closer to developers and prevent them from committing broken code. This is why we want to integrate our engine via API into all Integrated Development Environments.
Note: This document represents our vision of a perfect integration. Depending on specific IDE's limitations not all of these points may be possible.
The basic idea is that this extension collects the required files and sends them to our API. Afterwards, it receives structured data with suggestions. And then it shows those suggestions to the user inside of the IDE. Plus, it also allows users to ignore specific suggestions or entire folders/files. The ultimate goal is to achieve User Experience which is as simple as with linters.
Table of Content
- Package structure
- Meta files
- Configuration variables
- Login functionality
- User consent
- Integration with Deepcode API
- Ignoring files & folders
- New scan
- Extending bundle
- Handling file changes
- Analysing large folders
- Editor integrations/changes
- Code highlight
- Inline ignoring
- Analysis panel
- Summary line
- Online link
- Re-scan button
- Settings link
- Grouping and sorting
- File details
- Suggestion details
- Status bar
- Inactive plugin
- Loading state
- Zero problems found
Make sure you add a README.md file to the package's root folder. This file is addressed to future users of the package. Cover such aspects as installation, configuration and usage of the package. As a good example refer to Visual Studio Code extension.
All our extensions/plugins are open-sourced with License MIT. Please save it in the file "LICENSE".
To simplify debugging and publishing the package, please create a file "development.md" with instructions for developers. Make sure to cover the following topics:
- Local setup and installing dependencies
- How to run automated tests
- How to manually test this package before publishing
- How to publish it to the marketplace
Create a file CHANGELOG.md to track changes between releases. Follow these guidelines.
All IDE allow extensions to have custom settings to change their behaviour.
The base address of Deepcode API.
By default keep it empty, which means
Users, by changing this URL, can point it to their self-managed instance.
All analysis-related requests require user authentication. We want to allow users to manage their token from these settings.
Default: empty string.
Our engine can run linter analysis in addition to our own analysis. Use this flag as part of the analysis arguments.
Default value: False
If enabled, log important actions to the IDE's console.
Default value: False
We should trigger a login procedure, whenever our API returns 401.
First, we show a popup or overlay with prompt to login and a button/link. Once clicked, we should initiate the procedure of getting a new token. For this, user will need to login on DeepCode instance. In parallel, our extension starts a polling for the new token. Once received, save it in settings.
For legal reasons, we have to ask for the user's consent to analyze remotely their code.
As sometimes they don't realize that their code gets sent to our servers via Internet.
They might have signed an agreement prohibiting that.
Practically, we have to show a popup/overlay with the prompt "Confirm remote analysis of ( Terms and Conditions )" and a button "Confirm".
Once confirmed, save this consent for the specific project folder and don't ask for it in the future. If consent is not given, don't start the analysis.
The first step is to gather files for DeepCode's analysis. We call the set of files to be analyzed a
bundle. Not all files can be analyzed and some users may request to exclude some files from the bundle. Even though DeepCode API will exclude those files automatically, it's advisable not to send them at all.
Make a call to our API to receive instructions on what types of files DeepCode can analyze. The protocol is described here.
Users can exclude certain folders or single files by using special files. For optimal performance, we require for each integration to apply this logic and exclude those files from the bundle.
Handling file changes
Make sure that each time when user saves file(s), your extension sends those files for the analysis. For this, follow the guidance from previous step "Extending bundle". For better User Experience do it asynchronously and with some delay (of 3 seconds). During this delay, collect all changed files and send them all at once.
Analysing large folders
Our API servers have limitations for request size. Each request should not exceed 5MB ("this limit" later in this doc). Our protocol instructs to make 2 potentially large requests:
- File hashes
- File contents
To overcome this limitation, make sure you split your files in chunks and send them one after another using the protocol of Extending a bundle.
If you have separate files larger than the limit, please exclude them from your bundle. This is a known limitation of our system. But we also think that usually such files are production bundles, not the original files, so omitting them should be fine.
Our engine returns suggestions containing the locations of issues found in analyzed code.
Each extension should be able to mark those parts of the code. The highlighting color should depend on the severity of each issue. The severity is one of the fields in each suggestion. Where 1 is the least critical - called "info", 2 - "warning", 3 - "critical".
Example code highlighting in Atom editor for "info" suggestion:
For various reasons, users may want to ignore/disable specific suggestions given by our engine. Those cases include but not limited to:
- false positives (we aim to minimize those, but they still happen)
- personal preferences
To instruct the DeepCode engine to ignore certain suggestions, developers should include a line with a comment prior to a line with that issue.
This comment has a special format:
- DeepCode command:
deepcode ignoreto ignore only one occurrence, or
file deepcode ignoreto ignore all such issues within the current file
- Suggestion ID, for example
DisablePoweredBy(this ID is one of values of suggestion details returned by our API)
:right after suggestion ID, without space before it
- user comment, where the developer is able to give us some textual feedback and help us teach our engine
- in case of multiple issues on the same line, just repeat this template on the same line, divided by comma without space
Example for JS:
// file deepcode ignore DisablePoweredBy: some important reason, file deepcode ignore UseCsurfForExpress: another important reason
Implement functionality on adding such comments automatically. Ideally, upon creating the comment, we automatically select a user comment part and put a cursor at its beginning. This way we put more emphasis on it and collect more useful feedback.
This is how we do it in the Atom editor:
Keep in mind to respect language syntax. e.g // for JS/Java and
#for Python, so it's a good idea to use IDE API to add comment, instead of doing it yourself.
Each suggestion given by our engine has a useful description. One of the places were we want to show it is an overlay/popup - close to the affected code.
Example details overlay in Atom editor:
Each overlay should contain two buttons to compose the aforementioned ignoring comments for a specific instance and for a whole file.
Our goal is to show analysis results with our suggestions. All in one place. There are 2 major ways of achieving it:
- Integrate our suggestions into an existing "problems" panel;
- Build our own
Prior to choosing one or another, please analyze both possibilities and ideally present your outcomes to the DeepCode team to make a final decision.
In this document we assume building our own panel and therefore have complete freedom.
Please contact us at email@example.com to discuss it.
Example for Atom plugin:
On top of the panel, show a line following such a template:
Found problems: X in Y files. Critical: N, Warnings: M, Info: L
This text does not come from our API as is. Instead, we should combine it using structured data given by our analyzer.
On our website, we show more details and example fixes compared to what we can show in IDEs
Thus, it is advisable to add a link to current analysis results on our website. This link can be found in the API results.
In case of interruptions or any unexpected issues, users should be able to initiate a complete re-scan of their project.
Each IDE has a dedicated view for the extension's settings. On this page users should be able to change all the fields mentioned in "Configuration variables"
Example suggestions for Atom plugin:
Grouping and sorting
Our suggestions should be displayed as a tree with the following levels:
- File with issues
- Suggestion helpers
The files should be sorted comparing the following:
- Number of critical issues
- Number of warnings
- Number of info suggestions
Total number of all suggestions should not affect the order.
Suggestions inside each file also should be sorted by severity showing criticals first.
File line details
Each line for a file should consist of:
- File name
- File path
- Issues counter splitted by severity (with icons, e.g. look at the example above)
- Severity icon
- Full description
- An entire line should behave as a link. When clicked, the IDE should open related file and put a cursor at the issue's location.
- Two buttons to compose the aforementioned ignoring comments, same as on the "overlay popup"
Our extension should not be aggressive to users, and therefore, we suggest not to open our panel automatically. Only users themselves will do so by clicking on a designated area in IDE's status bar, which we need to develop.
Call this area "DeepCode". It needs to be clickable. On click, open our panel. Depending on the IDE, you may not be able to add your area, but are obliged to integrate into the "Problems" area.
Show splitted status on number of found issues (e.g. X critical, Y warnings, Z minor) with corresponding icons
Example deepcode area for Atom plugin:
Until the user completes both initial steps (login and folder consent), we should not send any requests to our API.
At this stage, please show "?" (questions mark) instead of numbers for critical and warning icons:
While sending files and waiting for results, please show a loading spinner.
Zero problems found
If the analysis returns zero suggestions, show a green checkmark instead of those numbers: