We address the need for precise and fine-grained code differencing tools in Solidity, the primary language for smart contracts on EVM-based blockchain platforms. SoliDiffy generates fine-grained edit scripts, which are crucial for tasks such code reviews and automated smart contract repair.
Browsable Diffs
You can browse through our diff pairs and their respective edit scripts generated by both SoliDiffy and Difftastic. Please keep in mind due to GitHub public API limitations, you can view the results of the first 1000 projects, and to view the rest you should manually clone the repository at https://github.com/SoliDiffy/SoliDiffyResults. The following form helps you interactively view the diffing results of the first 1000 projects (all project names starting with a, b, and c).
SoliDiffy Edit Script
Difftastic Edit Script
Research Questions
- RQ1: How does the performance of SoliDiffy compare to the most-closely related tool, Difftastic?
- RQ2: How does SoliDiffy perform when there are multiple changes in the smart contract source code?
- RQ3: How does the type of syntactic changes in file affect the performance of SoliDiffy?
- RQ4: How does SoliDiffy perform against line differencing on commit history of real-world smart contracts?
Results
RQ1: SoliDiffy consistently produces shorter edit scripts across a variety of projects compared to Difftastic. SoliDiffy generates fewer edit actions, demonstrating its advantage in handling complex modifications, and also successfully analyzed a higher percentage of diff pairs than Difftastic (96.1% vs. 95.1%).
RQ2: SoliDiffy consistently produces smaller edit distances than Difftastic, even as the number of mutations increases. SoliDiffy maintains a lower average edit distance regardless of the complexity of changes, while Difftastic exhibits higher variability in performance.
RQ3: SoliDiffy excels in handling complex syntactic changes, such as code block mutations, where it produces shorter, more meaningful edit scripts. Difftastic performs better in simpler cases, but struggles with complex structural changes.
RQ4: SoliDiffy effectively analyzes real-world commit histories, such as the Uniswap v4 core project. Compared to Git's line-based differencing, SoliDiffy provides more compact results, making it highly suitable for advanced smart contract analysis tasks.
