Setup practices detection with the AI Agent
Understand how your custom practices can be detected in your IDE, your CI/CD pipelines and code reviews, using the Packmind AI Agent.
Last updated
Was this helpful?
Understand how your custom practices can be detected in your IDE, your CI/CD pipelines and code reviews, using the Packmind AI Agent.
Last updated
Was this helpful?
The Packmind AI Agent relies on AI to generate a deterministic program to detect violations of a coding practice. (Think of it as a custom Eslint rule for instance, generated by AI).
To process a practice, the AI Agent will need at least:
A description for the practice (NB: that will be in most cases generated by Packmind AI)
A negative example.
As soon as a practice is validated, the AI Agent starts processing the practice.
The AI Agent defines a two-step workflow. You can configure and monitor it in the Detection page of a practice.
You can open the Detection page by clicking the Settings button in the Detection popover.
Detection guidelines, generated first by the AI Agent, indicate how violations of the practice can be detected in a source code using the code syntax. While the description of the practice focuses on explaining its intent, the guidelines only target the identification of its violation.
Once guidelines are generated, and every time they are updated, the AI Agent runs a new program generation.
The program must fulfill the following requirements to be successful:
All negative examples (added manually, unit tests, or generated by Packmind AI) must be detected by the program.
All positive examples (added manually, unit tests, or generated by Packmind AI) must NOT be detected by the program.
Unit tests are code snippets that either contain or do not contain a violation of a coding practice. This helps to document common cases where violations should be detected or not. It's helpful, for instance, when you've identified a false positive and want to document why this case should not be detected as a violation.
When you open the Detection page of a practice, you'll find a section called Unit Tests
In the Unit Tests section you can:
Add, edit, or remove unit tests.
Run the test suite to ensure that the program still behaves as expected.
When a new code example is added to an existing practice, Packmind’s analysis program may or may not detect it as a violation. If the program fails to detect the example, the associated practice will be marked as Outdated in the Packmind interface and a corresponding warning will be shown.
In this state:
the practice will no longer be detected in IDEs or via the CLI
developers won’t receive feedback based on it
To resolve this:
a Packmind user must regenerate the program for the affected practice
they must ensure that the new example is correctly identified as a violation after regeneration
Once the regeneration is successful, the practice will be detected again in the IDE and CLI tools as expected.
When you open the Detection page of a practice, you'll find a section called Detection Filters.
You can add one or several patterns both for inclusions and exclusions.
Packmind supports glob patterns, here are some examples:
**/*.spec.ts
Match files ending with .spec.ts
**/*Controller*
Match files which name containsController
**/infra/**/*.ts
Match.tsfiles under a directory named infra,directly or not (can be a sub-sub directory for instance)
**/infra/*.ts
Match .tsfiles directly under a directory named infra
apps/**/*
Match all files inside the root directory apps
If you cannot use AI in your context for compliance reasons, you can still use regular expressions or Semgrep patterns. Go to the Detection popover of a coding practice to open the configuration menu.
In the Practices section, a coding practice in Packmind has one of the several detection statuses:
To configure: The practice has no detection configuration yet.
Disabled: The practice has been manually disabled for detection
Outdated: The current program does not cover all unit tests and examples. This can happen when users update practice examples or unit tests. In this case, you can regenerate a new program.
In progress: The AI Agent is currently generating a program to detect violations of the practice.
Failed: The AI Agent could not succeed in generating a program. You'll have to ensure your practice examples are fine and refine the detection guidelines if needed.
Violations of practices are pushed to developers in 3 different channels:
Currently, the CLI supports two formatters for the output file:
The SonarQube generic format
The SARIF format (Static Analysis Results Interchange Format), which you can integrate with tools that support it
The answer is simple: no. The source code is sent through our plugins or the CLI through a secure SSL connection, but once our engine has analyzed the code, it's just not stored at all in our database. The database only stores source code related to a best practice description.
Make sure first in your Packmind organization to benefit from this.
When guidelines are generated the first time, few unit tests are also generated. These are useful to help the AI Agent to understand your practice (read more on unit tests ).
Keep in mind that you can edit both guidelines and unit tests at any time if you consider some adjustments are needed.
Ensure the detection guidelines and unit tests are correct - don't hesitate to be super accurate
Configured: The practice can be detected for violation
In IDEs, thanks to ;
During code reviews, thanks to our ;
When using the , locally or in the CI/CD process.
Please to suggest new formats.
