The valuable unit is not a repository snapshot. It is an issue, its context, the patch and the test that proved it, with every secret left behind.

A repository holds far more than source files. It can hold the cloud key someone committed in 2019 and "removed" in the next commit. Vendored dependencies with their own licenses. A customer's sample data pasted into a test fixture. And comments that explain why that bizarre-looking function absolutely must not be touched.

So preparing code datasets for AI takes more than running git clone on a snapshot. Your team has to define the task the data supports, preserve the relationships that make it understandable and establish what it actually has the right to license.

For many engineering archives, the most informative unit is not a directory. It is a complete issue-to-resolution story.

Illustration: a code diff beside a branch graph where tests pass and the fix merges. Conceptual artwork for this sponsored article.

Keep the issue, patch and test together

A hypothetical bug fix might include the reported behavior, the relevant repository version, the proposed change and the tests showing whether it worked. Strip out those relationships and a reviewer can see the solution but has no way to understand or verify it.

SWE-bench is a useful research reference here. It pairs real GitHub issues with repository context and evaluates fixes by running tests. A private archive still needs its own preparation and validation; resembling a benchmark certifies nothing.

Flag missing pieces explicitly. If the original environment cannot be rebuilt or the tests never ran, do not quietly label the example a verified successful repair.

The anatomy of a useful code example. A conceptual framework drawn from the article, not measured performance or a completed transaction.

Search history, not just the current branch

Deleting a credential from the current branch does not revoke it. GitHub advises revoking or rotating an exposed secret first. A properly revoked credential no longer grants access, while other sensitive content can remain in history, clones and forks and still needs release review.

Your export review should therefore cover history, branches, issues, logs and attachments within scope. Deleting a string from the visible working tree is not evidence that every copy is gone; clones and forks remember.

Keep remediation work separate from the dataset release. The release owner needs a record of what was excluded or transformed, not a note saying someone ran a scanner.

Review rights at the component level

Owning a repository does not mean owning everything in it. Third-party packages, contractor contributions, customer-supplied code and copied snippets can each carry their own obligations.

Identify the source and restrictions for the material you propose to include. Some components will need exclusion. Others will need extra documentation or review. Convenience must never set the rights boundary.

Understand the permitted AI uses and any exclusivity on the table, too. A license over selected engineering records can shape future commercial options even while the original repository stays internal.

Engineering acceptance check. Four review points from the article; the full text supplies the context and conditions.

Define an engineering acceptance check

Before delivery, a reviewer should be able to answer:

• Which repository version and issue does this example use?
• What information was available before the fix?
• Which tests or other evidence support the recorded result?
• Which secrets or restricted components were removed?
• What limits reproducibility?

And when you build an evaluation set, split related examples together. An issue and its near-identical twin fix landing on opposite sides of the train-test boundary is the kind of bug that silently inflates scores.

Where licensing fits

SourceX licenses approved business data, including suitable engineering histories, directly from companies for AI training and evaluation. Its process covers qualification, rights review, defined scope and preparation under a signed deal.

As for value, the advertised maximum of $4 million is the top end for the largest enterprise datasets, not a price per repository, a typical payment or a guarantee that any codebase will qualify.

What is a developer's first move?

Describe one potential task and map every record needed to understand it. Share that inventory with the authorized engineering, security and rights owners before anyone creates a release. A smaller engineering history that can be explained and checked is a far stronger starting point than a larger export nobody is willing to defend.

Key takeaway: The unit of value is the issue-to-resolution story, with every secret left behind.

If the company holds suitable engineering history, check company fit with SourceX before preparing a repository release. The initial assessment needs no source-code or issue-data uploads.

About the founder

Noah Loul is a four-time founder and the founder and CEO of SourceX, which licenses business data directly from companies for AI training and evaluation. He previously founded B2B Rocket, acquired by Blackpearl Group. His work focuses on building permission-based data infrastructure for AI.