Watch Joyce from Crossmint demo how we used Devin, Cognition Labs' autonomous developer, to scale GOAT - the open-source SDK for onchain AI actions
Background
Crossmint provides developer tools that help companies bring their applications, projects, and agents onchain. At the heart of our open-source initiatives is GOAT SDK, a community-owned library that enables AI agents to interact with blockchain tools—from making stablecoin payments to trading in decentralized markets, and more.
The Challenge
As maintainers of GOAT SDK, we faced a classic scaling challenge. The library requires constant integration work to keep pace with new protocols and dApps entering the ecosystem. Despite having an active community of developers, the integration backlog was growing faster than our contributors could handle. We needed a solution that could scale our development efforts while maintaining code quality.
Enter Devin
When Cognition provided us with Devin credits as part of their Open Source Initiative, we saw an opportunity to test an innovative hypothesis: Could we leverage AI to scale our open-source development while allowing human developers to focus on higher-order challenges?
Implementation and Results
The results have been remarkable. During our initial trial, Devin quickly became our #1 contributor by PR count (it merged 8 PRs, with the next contributor at 4). The majority of its contributions were high-quality and successfully merged. Two notable examples include:
- Adding support for DEXScreener plugin
- DEXScreener is a product with a REST API to check price and activity of crypto tokens
- We asked Devin to build an integration for GOAT with just a URL to their docs, and an instruction to make a plug-in.
- Later, we told Devin to look at some examples in the code to learn how to write it. In retrospect, we should have included this in the initial prompt, or in a playbook.
- Finally, we gave it one single round of feedback in GitHub, and this was enough to merge.
- Link to session
- Implementing Sui blockchain integration
- Sui is a blockchain, similar to Ethereum and Solana. This task was much harder than the above, adding a new chain required changing code across multiple layers, as well as adding examples.
- In this case, our prompt for Devin was more specific. And, we had to give 3 rounds of feedback. It got stuck in a bit of a loop because it used, for reasons beyond our understanding, an old version of Sui’s standard library.
- Nonetheless we were able to merge this with only an hour or so of human involvement, whereas doing it from scratch would have taken days.
- Link to session
What's particularly impressive is Devin's ability to work with minimal context—often requiring just a link to integration documentation to produce working code.
Key Learnings
Through our experience, we've identified three critical factors for success with Devin:
- Proper Training: Ensure Devin has sufficient knowledge of your repository and workflows when starting a new task. The keys to achieving this are: (1) doing this iteratively (starting with easier tasks and spending time after each to build Devin’s knowledge), and (2) patience: not assuming that if Devin made a mistake it won’t be able to learn from it.
- Clear Task Context: Provide sufficient background information and specific goals
- Validation Planning: Establish clear criteria for success upfront
Common Pitfalls to Avoid
The most common mistake we've observed is overestimating Devin's capabilities, setting unrealistic expectations and disappointing when it doesn’t work as expected. Developers have a tendency to approach Devin as a “superhuman” entity, which leads to:
- Providing insufficient context
- Assuming pre-existing knowledge of codebases
- Having unrealistic expectations
Future Outlook
While it’s still in early stages, we're optimistic about Devin's role in scaling our repository sustainably. This allows our human contributors to focus on critical areas where they add the most value:
- Architectural improvements
- Developer experience enhancements
- Community engagement
- Strategic prioritization
By leveraging Devin for routine integration work, we're creating a more efficient development ecosystem that benefits both our maintainers and the broader open-source community.
