1 Demystifying the DAO Governance Process Junjie Ma, Muhui Jiang, Jinan Jiang, Xiapu Luo, Yufeng Hu, Yajin Zhou, Qi Wang, Fengwei Zhang Abstract—Decentralized Autonomous Organization (DAO) be- to the absence of clear explanations describing the code to be comes a popular governance solution for decentralized appli- executed in the proposal. An example is the Beanstalk attack, cations (dApps) to achieve decentralized governance. In the leading to a loss of 182 million dollars [18]. The attacker DAO, no single entity can arbitrarily control the dApps without deceived members into trusting the code in the proposal was approval from the majority of members. However, despite its advantages, DAO has also been targeted by several attacks, benign. In reality, the code intended to transfer all assets leading to the loss of millions of dollars. In this paper, we first owned by Beanstalk to the attacker. Moreover, DAOs can providedanoverviewoftheDAOgovernanceprocesswithinthe also be manipulated by the developers. Normally, the dApps’ blockchain.Next,weidentifiedtheissueswithinthreecomponents tokencontractshouldbecontrolledbythegovernanceprocess, of governance process: Governance Contract, Documentation, ensuring that without the majority’s permission, no one can and Proposal. Each of these components is vulnerable to issues that could potentially result in substantial financial losses. Then transfer the locked tokens in the contract. However, in some we developed automated methods to detected above issues. To DAOs, the contracts are controlled by specific developers. investigate the issues within the existing DAO ecosystem, we This allows the developers to arbitrarily control the contract constructedastate-of-the-artdatasetthatincludes16,427DAOs, withoutobtainingpermissionfromthegovernanceprocess.An 183 documentation, and 122,307 proposals across 9 different example of this is the VPANDA DAO Rug Pull [19], where blockchains. Our analysis reveals that a majority of DAO developers and members have not given sufficient attention to thedeveloperillegallytransferredover1millionlockedtokens these issues, especially in the area of proposal. The result shows fromthetokencontracttoswapforover265thousanddollars, that over 60% of the examined proposals fail to provide a resulting in a 99% drop in the token value. consistent description and code for their members, highlighting Previous studies within the field of DAOs have predomi- a significant gap in ensuring transparency within the DAO nantly focused on analyzing DAO activities and issues related governance process. For a better DAO governance ecosystem, DAOdevelopersandmemberscanutilizethemethodstoidentify tovotinginthegovernanceprocess[20],[21],[22],[23],[24], and address issues within governance process. [25], [26], [27], [28], such as centralized voting power. To the best of our knowledge, there is no work focusing on the Index Terms—Decentralized Governance, Program Analysis, Smart Contracts, Language Models. issues of the entire DAO governance process. Our work fills this gap by conducting a comprehensive study and answering the following 3 research questions. Each research question I. INTRODUCTION is related to a distinct aspect within the governance process DECENTRALIZED Autonomous Organization (DAO) is identified in the section III. a governance method constructed based on blockchain •RQ1: Do existing DAOs achieve fairness decentralized gov- smart contracts [1]. The DAO ensures that all privileged ernance? actions must gain the majority of member consensus, thus •RQ2: Do existing DAOs offer sufficient governance process effectivelypreventingarbitraryactionsfromacertainmember. documentation for their members? Recently,asignificantnumberofdecentralizedapplications •RQ3: Do existing proposals ensure consistency between (dApps) have adopted DAO as their governance method. For descriptions and code? example,Uniswap[2],oneofthemostvaluableDecentralized For RQ1, we verify that the DAO achieves decentralized Exchange (DEX), with a daily trading volume exceeding 500 governance, ensuringdevelopers can not compromisethe fair- million dollars [3], employs DAO for its asset management. ness of the governance process. We first adopt static analysis Additionally, DAO platforms such as XDAO [4], Aragon [5], of the governance contract to ascertain whether it has the and DAOhaus [6], which help developers to deploy DAO in requiredgovernancefunctions.Then,weextractthecontroller minutes, have attracted the interest of thousands of organiza- addresses of privileged functions to determine whether the tions [7]. In particular, XDAO [4] has facilitated the setup of governance contract is self-governed or controlled by devel- over 33,000 DAOs across various blockchains. According to opers. Lastly, we trace the creation process of the gover- the analysis [8], the total treasury governed by DAOs exceeds nance contract to ensure that developers can not arbitrarily 18.8billiondollars,withover2.5millionusersparticipatingin modify the contract’s code logic. For RQ2, we investigate DAOgovernance.ThistrendindicatesthatDAOhasbecomea whether the DAO offers adequate guidance to its members populargovernancemethodadoptedbyblockchaindevelopers. forparticipatinginthegovernanceprocess,therebymotivating However, this rapid increase in DAOs has come with memberengagementingovernanceprocess.WeemployLarge challenges. Many DAO developers, as well as members, fail Language Model (LLM) with Chain of Thought (CoT) [29] topaysufficientattentiontotheissuesintheDAOgovernance to evaluate if the DAO documentation complies with the six process. This oversight has led to an increase in attacks requirements outlined in the DAO Model Law [30]. In RQ3, targetingDAOs[9],[10],[11],[12],[13],[14],[15],[16],[17], weassesswhethertheproposalssubmittedbymembersexhibit [18].Forinstance,thegovernanceprocesscanbeattackeddue consistent and immutable code behavior, aligning with their 4202 raM 81 ]ES.sc[ 1v85711.3042:viXra 2 descriptions. Thus, attackers cannot disguise malicious pro- II. BACKGROUND posals as legitimate ones to misappropriate the DAO’s funds. A. Decentralized Autonomous Organization Initially, we trace the proposal code to verify its immutability Decentralized Autonomous Organization (DAO) in after submission. Subsequently, we employ a combination of blockchain is first introduced by Ethereum white paper [1]. Natural Language Processing (NLP) and LLM to ensure that DAOutilizessmartcontractstoenablecollectivecontrolofthe all actions prescribed by the code are accurately reflected in organization by all its members. Using smart contracts in the the proposal descriptions. The issues pertaining to DAOs, as DAO allows for establishing organization rules and managing discussed in our study, have not been previously explored the treasury through immutable code. Currently, there are in other research. Furthermore, our investigation encompasses two types of DAO governance [32][33]: on-chain governance an extensive collection of over 16,000 DAOs from 5 distinct and off-chain governance. On-chain governance requires all platforms and 9 different blockchains. the governance processes to be conducted on the blockchain Our results show that not all DAO governance processes by smart contracts, including proposing proposals, voting, are precisely implemented. We identify one DAO in which and executing. On the contrary, in off-chain governance, the the governance contract can be destructed and redeployed at decision-making process (e.g., proposing proposals or voting) thesameaddressbythedeveloper.Intermsofprovidingdocu- is performed outside the blockchain. The execution process mentationtoassistmembersinparticipatinginthegovernance is carried out manually by the DAO developer, granting process, over 98% of DAOs fail to provide such documen- it complete control over the DAO contracts. We exclude tation. Given that such documentation is intended to equip off-chain governance from our scope as off-chain governance members with essential information for DAO governance, its contravenes the requirements in the DAO definition [1] and absence can deter member participation in governance or the DAO Model Law [30], which mandate governance to be lead to controversial voting results. For these proposals in executed on smart contracts. the governance process, we find that only 34% of proposals (8,584) contain descriptions about the code actions in the B. DAO Platform proposal. Furthermore, of these proposals, less than 30% The DAO platform is designed to provide DAO developers (2,536 proposals) provide a complete explanation of the code, with the tools to easily create their own DAOs. Developing a such as what function will be invoked and how many tokens DAO requires advanced programming and blockchain knowl- willbetransferred.Toassesstheeffectivenessofourapproach edge. Current DAO platforms such as XDAO [4], Aragon [5], in detecting real-world governance attack cases, we evaluate DAOhaus [6], and DAOstack [6] offer comprehensive assis- ourapproachagainstthelatest13governanceattackincidents. tance in DAO creation. Their assistance spans a wide range, Ourapproachsuccessfullydetectsalloftheseattacks.Wehope from on-chainsmart contracts deploymentto the creationof a our paper can guide developers in deploying and maintaining dedicatedvotingwebsite.Inthesecases,developerscancreate their DAOs in a more comprehensive and secure manner and their own DAO in minutes. also enlighten members about potential risks within the DAO governance. C. DAO Model Law Our contributions can be summarized as follows. The DAO Model Law [30] is a type of Model Law [34], •Public Dataset: We collected 16,427 different DAO imple- seeks to bridge the divide between DAOs and traditional mentations, 183 documentation, and 122,307 proposals across regulatoryframeworksyettoadapttonewcompanystructures 9 popular blockchains. Our dataset included famous DAOs fostered by the blockchain. DAO Model Law stipulates rules such as Uniswap [2] and Compound [31], as well as DAOs applicable to both the on-chain smart contracts and off-chain from platforms like Aragon [5]. The collected data will be documentation. Once these rules are met, DAOs and their released for further research. members can achieve legal certainty. The DAO Model Law •Comprehensive Study: We conducted a comprehensive is the only document that provides rules for smart contracts study on a large amount of DAO implementations, gaining and documentation. an in-depth understanding of the issues addressed in the 3 proposed research questions concerning the DAO governance III. DAOGOVERNANCEPROCESS process. Our findings revealed that over 99% of DAOs failed We provide a comprehensive overview of the DAO gover- to provide documentation. Besides, over 90% of the existing nance process, as shown in Figure 1. proposals failed to elucidate the code actions. Participants.TheparticipantsintheDAOgovernanceprocess •Insightful Findings: We found the current DAO imple- fall into one of three roles: developer, member, and guardian. mentations have many security issues, which deserve our The first role, developer, is involved in the development of attention. 1) More than 600 DAOs contain privileged func- the DAO’s smart contracts and interface. He is responsible tions controlled by unidentified entities, potentially serving for ① deploying the governance contracts to the blockchain as backdoors that could compromise the governance process. network,aswellas②creatingthedocumentationfortheDAO. 2) The governance contract code logic can be arbitrarily The second role, member, is a blockchain user who learns modified without changing its address, allowing developers the governance process by ③ reading the documentation. He to manipulate governance results. can participate in DAO governance by ④ submitting or ⑤ 3 Fig. 2. The decompiled governance contract from mini dao shows that the developer controls privileged functions (setVotingPeriod and setPropos- alThreshold), enabling him to control proposal voting duration and required votingpower. This encompasses delivering detailed information on how Fig.1. TheDAOGovernanceProcess. to become a DAO member, providing step-by-step guides to participate in the governance process, and outlining the existence of guardian. voting for a proposal via the DAO governance contract [31]. The last role is the guardian, a specific blockchain user taskedwith⑥monitoringtheDAOgovernanceprocesswithin IV. APPROACH A. Research Questions the governance contract. If the guardian detects malicious proposals targeting DAO governance, he has the authority We examine the issues within each component of the within the governance contract to cancel such proposals. governanceprocess-GovernanceContract,Documentation, Governance contract. The governance contract controls the and Proposal -with the following research questions. governance process, storing all the proposals and votes from The Governance Contract controls the entire governance members. It provides functions that allow members to submit process. Therefore, according to the definition [1], it is es- new proposals, vote on these proposals, and execute the sential for achieving fairness in decentralized governance, code within the proposals. The governance contract should preventing developers from arbitrarily manipulating the re- be configured as the only way to change the parameters of sults. However, a malicious developer could embed privileged DAO contracts. functionswithinthegovernancecontractthatarecontrolledby Proposal. Proposal refers to a formal submission to gover- themselves. This manipulation allows them to undermine the nance contract that is made by a member to suggest changes governance process, compromising the fairness of proposal to the DAO (i.e., funding request, contract parameters config- outcomes. For instance, functions such as setVotingPeriod uration).Typically,asshowninFigure9,theproposalencom- and setProposalThreshold within the governance contract are passes two elements: description and code. The description, designed to set the proposal voting duration and the required pennedinnaturallanguage,outlinestheintentoftheproposal. votingpowerforaproposaltopass.Normally,thesefunctions It provides members with information regarding the proposal can only be called by the governance contract itself, ensuring code as well as the reason behind it. The code contains the thatonlypassedproposalscanmodifytheseparameters.How- code that will be executed by governance contract if the ever, in the case of the governance contract 0x41E6......7a42 proposal gets passed. It refers to the technical implementation from the DAO mini dao shown in Figure 2, these functions of the proposal. are controlled by an admin, which is an Externally Owned Governance process. Managing and implementing changes Account (EOA) specified by the developer, rather than the within a DAO relies on the governance process. This is governance contract itself. Thus, the developer can pass any achieved by submitting proposals to the governance contract proposal by adjusting the voting delay to allow only them- and conducting votes on these proposals. If a proposal passes selves to vote, or cancel any proposal by setting the proposal the voting process, the code within it is executed by the gov- threshold to a high value. ernance contract to implement the changes towards the DAO. Thus, we propose RQ1 to examine whether the existing This ensures all the changes are approved by the majority Governance Contracts belonging to DAOs achieve compre- of the DAO members. The governance process begins at a hensive in decentralized governance. member ④ submitting a proposal to the governance contract. RQ1: Do existing DAOs achieve fairness decentralized Then, a member can ⑤ cast vote for the newly submitted governance? proposal. A proposal is passed when it has received sufficient As for the Documentation, each DAO should provide voting power in support from members. If the guardian does detailed documentation for their member instructs members notidentifytheproposalasamaliciousone⑥,thecodewithin on participating in the governance process, emphasizing the the proposal will be executed by the governance contract ⑦. disclosure of its critical aspects. For example, as illustrated Documentation. Besides the governance contracts, the DAO in Figure 3, the DAO Compound offers comprehensive doc- also needs to provide the documentation. Considering the umentation for its members, detailing how to engage in the complexity of the governance process, the documentation governance process. The lack of governance documentation should provide complete guidance on governance process. can hinder members engagement in governance, as members 4 Fig. 4. The malicious proposal in the YAM governance attack deceived memberswithadescriptionfromapreviousproposal,claimingitwouldreturn rewardstotheDAO.However,theactualintentofthecodewastotakecontrol ofthegovernancecontract. TABLEI TYPESOFDAOS,ALONGWITHTHEIRCORRESPONDINGQUANTITY, WEBSITE,DOCUMENTATION,ANDPROPOSALINTHEDATABASE. DAOType DAO(total) DAO(used) Website Documentation Proposal XDAO 33,685 14,415 336 38 97,156 Fig. 3. The Compound governance documentation provides DAO members Aragon 2,384 1,182 111 44 16,149 Tally 615 544 144 92 3,953 withguidanceonhowtovoteforproposals. DAOhaus 243 221 7 1 1,712 DAOstack 219 62 21 5 2,190 Self-developed 3 3 3 3 1,047 Total 37,149 16,427 622 183 122,207 need to read the governance source code to learn how to participate in the governance process. This scenario could resultinasituationwheregovernanceresultisdeterminedand dataset is listed in Table I. Specifically, we collect DAO in- controlled by only a handful of members. For example, in the formation, websites, documentation, and proposals. However, Synthetify DAO governance attack on 17 October 2023 [35], the collection of these data presents several challenges. First, an attacker submitted a malicious proposal aiming to seize there is a lack of comprehensive dataset that encompasses all control of the DAO’s assets. Due to the lack of governance information related to DAOs. Second, current DAO informa- documentation, none of the DAO members actively partici- tion does not include DAO documentation and only provides patedinthegovernanceprocess,andasaresult,noonevetoed afewDAOwebsites.Third,notallDAOplatformsofferAPIs this malicious proposal during the 7-day voting period. This to fetch proposals. To solve these challenges, we outline the oversight led to a loss of 230 thousand dollars. method for each type of data as follows. We assess in RQ2 whether DAOs provide sufficient docu- DAO information. We locate the DAO information mainly mentation on governance processes for their members. from previous studies [36], [26] and available DAO datasets. RQ2:DoexistingDAOsoffersufficientgovernanceprocess Specifically,wegatherDAOinformationfrom1DAOdataset, documentation for their members? DeepDAO [8], and 5 DAO platforms (i.e., XDAO [4], AsfortheProposal,attackerscansubmitmaliciouspropos- Aragon [5], DAOhaus [6], DAOstack [37], and Tally [38]). als to gain control of the DAO or misappropriate the DAO’s Note that our data focuses on EVM-compatible chains whose assetsbyembeddingmaliciouscodewithintheproposal.They Total Value Locked (TVL) is larger than 50 million dollars. may deceive members into believing the code is legitimate by We collect DAO information from these sources until Feb 1, providing a misleading description. For example, during the 2024. governance attack on the DAO YAM on July 9, 2022 [9], the To filter out unused DAOs from these DAO sources, which attacker proposed a proposal shown in Figure 4. The attacker mightcausebiasintheoveralltrustworthyresults,weexclude misled members with a description from a previous proposal, DAOs that have never proposed any proposals. claiming it aimed to return rewards to the DAO. However, the We acquire the DAO name (or id), governance contract codeactuallytransferredownershipofthegovernancecontract address, and website (if available). Since data collected from fromtheDAOtotheattacker,resultinginalossof3.1million DeepDAO includes DAOs also present in the other platforms, dollars once it succeeded. we remove DAOs from DeepDAO that are duplicated in other In RQ3, we examine the consistency between the proposal data sources. Note that two DAOs with identical contract descriptions and the code to prevent malicious members from addresses are treated as the same one. Finally, we collect 3 submittingproposalsthatdeceiveothermembersbydisguising distinct DAOs from DeepDAO, which are not included in the malicious intentions as legitimate actions. other 5 platforms and are marked as independent by Deep- RQ3: Do existing proposals ensure consistency between DAO.Welabelthese3DAOsasself-developed.Toensureour descriptions and code? datasetiscompleteenough,weverifyitscontentswiththetop 20 DAOs listed on CoinMarketCap[3]. The result shows that B. Data Collection allthetop20DAOs,includingUniswap[2]andCompound[31] In this section, we collect data related to DAOs to address from Tally, as well as Curve[39] and MakerDAO[40] from the research questions mentioned above. The result of our DeepDAO, are included. 5 Documentation. Collecting the corresponding documentation contractsprovidedbytheplatformfacilitatedecentralizedgov- is not easy as the aforementioned dataset usually do not ernance and the corresponding DAOs adopt the same gover- provide such resources directly. To address this, our initial nancecontractasprovidedbytheplatform,wedeterminethat stepinvolvessearchingfortheDAOwebsitewithinthedataset such a DAO achieves the decentralized governance. To verify or its associated DAO platform. Should this approach prove whether a DAO’s governance contract matches the template unsuccessful, we try to query its public name tag [41] from governance contracts, we first trace the creator of the DAO blockchain scanner to find whether the governance contract governance contract by obtaining the governance contract haslinkedtotheDAOwebsite.Then,weutilizeSelenium[42] creation transaction from the corresponding blockchain. If the to crawl through the DAO website to gather the documenta- creator address of the DAO governance contract is identified tion. Specifically, we focus on links containing the keywords as the deployer address listed in the platform’s deployment whitepaper or doc. If such specific links are not found, we guidance,weconfirmthatthegovernancecontractisthesame archive the entire website. asthetemplateprovidedbytheplatform.Forthosegovernance Proposal. To retrieve the proposals, we utilize the inter- contracts whose creators differ from the platform deployer, faces provided by platforms such as Aragon, DAOhaus, and we download the bytecode of the DAO governance contract DAOstack to download all the proposals corresponding to and the corresponding template contract from the respective each DAO. If these platforms do not provide the proposal blockchain. We then compare them to ascertain whether they information for the DAO, we retrieve the event logs from the are identical. DAO’s governance contract address and extract the proposal For DAOs from Tally, the developers are allowed to add information inside the logs. new functions based on the template contract provided by OpenZeppelin [43] or Compound [31]. We can not directly V. DOEXISTINGDAOSACHIEVEFAIRNESS compare the bytecode of these contracts to ascertain if it is DECENTRALIZEDGOVERNANCE?(RQ1) the same as the template contract. We check whether the governance contract includes the three governance functions In this section, we examine whether existing governance fromthetemplatecontract(i.e.,Propose,Vote,andExecute)as contracts have implemented fairness in decentralized gover- required by the DAO Model Law [30] as well as the template nance. Specifically, we assess three aspects of the gover- contract from OpenZeppelin and Compound. (1)Propose. A nance contract: soundness, independence, and immutability. member can submit a proposal by invoking this function. Forsoundness,weverifywhetherthegovernancecontracthas (2)Vote.Foraproposalrecordedinthecontract,membershave the capability to achieve the intended governance processes. the ability to cast their votes using this function. (3)Execute. The absence of such capability constitutes a violation of the The function can execute the code of the proposal. If the requirements of a DAO. Secondly, we examine independence DAO’s governance contract includes all three aforementioned by ensuring that the governance process is controlled by functions, we deduce that it adheres to the template contract. the governance contract. This ensures that developers are Todeterminewhetherthetargetgovernancecontractpossesses preventedfromcompromisinggovernanceoutcomesbyinvok- the required functions, we compute the similarity between ing privileged functions. Lastly, regarding immutability, we the functions in the governance contract and the required investigate whether the governance contract’s code logic can functions. be altered by developers, which could lead to manipulation of For the rest of DAOs, if the governance contract is open- governance and misappropriation of assets from members. source and the documentation supports decentralized gover- nance while also providing its address, we infer that such A. Soundness of Governance Contract a DAO has achieved decentralized governance. Otherwise, As stipulated by the DAO definition [1], [30], the gov- we check whether the governance contract is similar to the ernance of a DAO must be decentralized. This necessitates contractprovidedbytheplatformorcontainssimilarfunctions that the DAO achieves decentralized governance through its from the template contract. governance contract. Therefore, if the governance contract To compute the similarity between contracts, we follow the lacks the capability to facilitate decentralized governance, it methodologyproposedbypreviousstudy[44].Specifically,we constitutesaviolationofthefundamentalprinciplesofaDAO. extract the bytecode of the contracts and eliminate the param- Approach. To evaluate whether a DAO has soundly im- eters of opcode PUSH. Therefore, we compute hypervectors plemented decentralized governance within its Governance of n-grams (n=5) of opcodes for each contract. The similarity Contract, we employ different methods for different types of scoreiscalculatedbytheJaccardsimilarityoftheirrespective DAOs. hypervectors. If the similarity score exceeds the threshold, we For DAOs from platforms XDAO, Aragon, DAOhaus, and determinethatthetwocontractsaresimilar.Weadheretoprior DAOstack, it is mandated that they utilize the template gov- research by setting the threshold value at 0.8 [45]. ernance contracts provided by the respective platforms [21]. To determine the similarity between functions, we utilize Initially, we conduct a manual analysis to verify whether EVMCFGBUILDER[46]toextracteachfunction’sbytecode the contracts provided by these platforms soundly implement from the contract. We adopt the same method used for decentralized governance. Subsequently, we ascertain whether computing the similarity scores between contracts to compute a DAO belonging to the aforementioned types adopts the thesimilaritybetweenfunctions.Inordertomitigatetheeffect provided governance contract. Therefore, if the governance of different Solidity versions causing discrepancies in the

6 TABLEII NUMBERSOFDAOSTHATACHIEVEDECENTRALIZEDGOVERNANCE (DG),ALONGWITHTHOSEWHEREPRIVILEGEDFUNCTIONSINTHE GOVERNANCECONTRACTARECONTROLLEDBYTHEGOVERNANCE CONTRACTOROTHERENTITIES. DAOType WithDG WithoutDG Governance Other XDAO 14,415 0 14,296 119 Aragon 1,182 0 775 407 Tally 544 0 515 29 DAOhaus 221 0 221 0 DAOstack 62 0 62 0 Self-developed 3 0 3 0 Total 16,427 0 15,872 555 Fig.5. Asimplifiedprivilegefunctionrestrictionrequiresthefunctioncaller tobethegovernancecontract. bytecode pattern, we compile the contracts with each major proposal. This could be achieved by arbitrarily adjusting the Solidity version. If the target function matches any version of voting period and the required voting power needed to pass thetemplatecontractfunction,wedeterminethefunctionsare the proposal. similar. Approach. For DAOs from platforms XDAO and Aragon, Result. The results, as shown in Table II, reveal that all the theseDAOsadoptthesamecontractfortheirgovernancelogic used DAOs achieve decentralized governance. and access control. Meanwhile, these platforms also provide To verify the effectiveness of our approach, we extend it an official API [4][5] to query the governor of the privileged to those DAOs that do not have proposals. Among 20,722 functions. Thus, we can determine whether the governance unused DAOs, we detect 303 DAOs that do not achieve contract governs all privileged functions by querying with the decentralized governance, including 193 from Aragon and provided APIs. 109 from DAOstack. We further conduct a manual analysis For the others, inspired by previous studies [47][48], we on these 303 DAOs to identify any false positives and de- apply static analysis of the governance contract bytecode to termine their causes. Within the 194 DAOs associated with identifyprivilegedfunctionsandextracttheprivilegedaddress the Aragon platform, we find that they do not provide the fromthesefunctions.Inparticular,toidentifyprivilegedfunc- governance contract address. This issue is attributed to the tions, we investigate whether a function requires the caller’s DAO developers’ decision to omit decentralized governance address, obtained from the opcode CALLER, to compare support during the creation process via Aragon. Similarly, with a specific address from contract storage by the opcode for the 109 DAOs on the DAOstack platform, we discover EQ. The result is used to determine the jump target. We that they do not support decentralized governance due to then extract this address and compare it with the governance their original design. There do not exist any false positives. contract address to ascertain if they align. These observations underscore that DAO platforms provide Result. As demonstrated in Table II, the majority of DAOs, developers with the discretion to either incorporate or exclude particularly those on platforms like DAOhaus and DAOstack, decentralized governance during DAO creation. To adhere the strictly adhere the requirement that all functions within the principles of decentralization, DAO platforms might consider governance contract should be governed by the governance makingdecentralizedgovernancesupportamandatoryfeature contract itself. for developers. However,555governancecontractsretaincertainprivileged functions that are not governed by themselves. As indicated B. Independence of Governance Contract by the results in Section VI, most DAOs fail to explain the existence of guardians. Thus, it is hard for members to All privileged functions within the governance contract classify whether these functions are potentially backdoors or should be controlled by the governance contract itself to designated for guardians to protect the governance process. ensure the governance process is independent and protected from any potential violations by developers. The privileged C. Immutability of Governance Contract function is defined as a function that can be executed only by a privileged address [47], [48]. As shown in Figure 5, After the Constantinople update [49], the EVM introduced the setVotingDelay function, designed to adjust the voting a new opcode, CREATE2, which allows a smart contract time within the governance contract, has a modifier named to be deployed at a predetermined address [50]. This can onlyGovernance. This modifier specifies that only when the be exploited as an attack vector since it allows contract functioncaller’saddressmsg.sender matchesthegovernance developerstomodifythecontractcodeafterdeploymentwhile contractitself,thefunctioncanbeinvoked,indicatingthatthis maintaining its address unchanged [13], [51]. Compared to function is under the governing of the governance contract. the normal contract creation opcode CREATE, where the However, if certain DAO functions are not governed by the contractaddressiscomputedaskeccak256(address+nonce), governance contract, this could lead to security vulnerabili- address represents the contract creator’s address, and nonce ties, as illustrated in Figure 2. An attacker could manipulate denotes the total transaction number of the contract cre- the governance process by controlling the outcome of any ator. In CREATE2, the contract address is calculated as 7 keccak256(0xff +address+salt+keccak256(CODE)). deployment transaction. If the CREATE2 opcode is used Here, address refers to the contract creator’s address, salt is to create C , we deem that C is created by CREATE2. i i a 32-byte string, and CODE corresponds to the init code of We take advantage of the Tenderly API [52] to access the thenewcontract.Thisintroducesapotentialrisk,asitpermits sequenceofexecutedopcodesfromthedeploymenttransaction developerstoarbitrarilymodifythecontractcodewhilemain- of contract C . Finally, if we determine that contract C is i i taining the same contract address by the CREATE2. How- created by CREATE2 and all preceding contracts within ever, the predetermined contract address from CREATE2 the chain can destruct themselves, our algorithm returns true; can not already have a contract code. This limitation can be otherwise, returns false. circumvented using the opcode SELFDESTRUCT, which Result.WeidentifiedoneDAOfromTally,associatedwiththe can destroy a contract along with its code. governance contract 0xe28...d5. This contract is created by Thus, the attack can be launched with the follow- CREATE2andutilizestheDELEGATECALLopcodeto ing steps. An attacker X first utilizes CREATE2 interface with external contracts. Notably, such contracts can to deploy contract A at the address calculated as bedestroyedbydevelopersandsubsequentlyredeployedatan ADDR(A) == keccak256(0xff + ADDR(X) + 0x0 + identical address. keccak256(CODE(A))). Thus, X uses contract A to deploy Answer to RQ1: Among the 16,427 DAOs analyzed, 665 thegovernancecontractGusingtheopcodeCREATE atthe could be manipulated by developers. Besides, DAOs listed address as ADDR(G) == keccak256(ADDR(A)||0). To by DAO platforms can not be considered trustworthy, as changethecodeofthegovernancecontractG,theattackerfirst these platforms do not mandate the actual achievement of destroys contracts G and A using the SELFDESTRUCT decentralizedgovernanceandlackthemechanismstoverify opcode. The attacker then deploys a new contract A′ by theconsistencyofaDAO’sgovernancecontractwithitsin- CREATE2, using the same salt and contract code as tendedtemplatecontract.Furthermore,thereexistsoneDAO A. Given that A′ and A share identical salt and contract whose governance contract code logic can be manipulated code, the address of A′ will be the same as that of A: by the developer using the CREATE2 opcode. ADDR(A′) == keccak256(0xff + ADDR(X) + 0x0 + keccak256(CODE(A))) == ADDR(A). In the final stage, the attacker deploys the new governance contract G′ using A′ and the CREATE opcode. As A′ is a newly deployed con- VI. DOEXISTINGDAOSOFFERSUFFICIENTGOVERNANCE PROCESSDOCUMENTATIONFORTHEIRMEMBERS?(RQ2) tract,itsnonceremains0,leadingthenewgovernancecontract G′ to have the same contract address as G: ADDR(G′)== The documentation is expected to provide a complete keccak256(ADDR(A)||0) == ADDR(G). Thus, despite overview of the DAO, elaborating on the governance process beingcreatedbyCREATE,contractG′ hasthesameaddress interactionsforitsmembers.ConsideringthattheDAOModel as G, but with a different code. Therefore, it is essential Law[34], as referenced in Section II-C, prescribes specific to ensure the governance contract is not at risk of arbitrary rulesfordocumentation,wehaveconsolidatedthedocumenta- modifications to its contract code via CREATE2. tionrequirementsfromtheDAOModelLawintothefollowing Approach.WefirstdefinetheContractCreationChain(CCC) sixrules.1)MemberParticipation.Thedocumentationshould of a governance contract as follows: Given a governance con- provide guidelines on how blockchain users can become tract address G, we trace its contract deployment transaction. DAO members and participate in governance, as well as the If it is deployed by a contract C , we add it to the CCC. We participation rights in the governance process. 2) Member 0 then trace the creator of C , designated as C , and continue Exit. Apart from participating in DAO, the documentation 0 1 thisprocessuntilwefindacontractthatiscreatedbyanEOA should also describe the steps a member needs to follow to addressE.TheCCC(G)=<G,C ,C ,......,E >showsthe exit the DAO, whether in a voluntary or involuntary way. 3) 0 1 governance contract G is created from a chain of contracts Voting Power. The documentation should clearly explain how that extend from C to E. voting power is calculated and distributed among members, 0 The procedure to determine whether a governance contract as voting power determines the weight of a member’s vote. is at risk from CREATE2 is shown below. For a given Failing to explain voting power could discourage member governance contract address G, we first construct its CCC. participation in voting or, conversely, enable a member to ac- To do this, we use the corresponding blockchain scanner cumulate excessive voting power, potentially allowing him to to query the respective contract deployment transaction. Af- arbitrarily control the result of voting. 4) Minority Protection. ter building the CCC, For each contract C in CCC(G), The documentation should explicitly state if it includes any i we first investigate whether the contract can destruct it- provisions for protecting the minority rights of its members. self via the opcode SELFDESTRUCT to erase its own This is crucial because minority members may need to raise code. However, a potential attacker could conceal the opcode disputes against specific decisions, particularly in situations within a different contract and indirectly execute it using the where a single member controls the majority of voting power. DELEGATECALLtodestroytheoriginalcontract.Hence, 5) Governance Process Guide. A detailed guide to the gover- if a contract contains the opcode SELFDESTRUCT or nanceprocessisnecessaryformembers.Forinstance,thestep- DELEGATECALL, we conclude that it can destruct itself. by-stepinstructionsforsubmittingproposalsandcastingvotes. Subsequently,inordertocheckwhethercontractC iscreated 6) Appointment of Guardian. The appointment of a guardian i byCREATE2,wetracetheopcodesusedduringthecontract is crucial to alleviating security concerns among members. 8 TABLEIII EVALUATIONOFCHECKINGWHETHERTHERULEISMENTIONEDINTHE DOCUMENTATION. ChatGPT[53] Claude[59] RuleName Recall Precision F1-score Recall Precision F1-score MemberParticipation 0.69 0.95 0.80 0.74 0.95 0.83 MemberExit 0.00 0.00 0.00 0.00 0.00 0.00 VotingPower 0.81 0.87 0.84 0.78 0.48 0.59 MinorityProtection 1.00 1.00 1.00 1.00 0.50 0.66 GovernanceProcessGuide 0.68 0.90 0.77 0.92 0.80 0.86 AppointmentofGuardian 0.89 1.00 0.94 0.89 0.73 0.80 Given the significant privileges the guardian holds, such as controlling the privilege functions in the governance contract, their role should be disclosed in the documentation. Considering most of the members are not able to reliably Fig. 6. The abbreviated question chain to query whether the 6 rules are and accurately extract information from the on-chain DAO mentionedinthedocumentation.EacharrowrepresentsaYesresponsefrom ChatGPT. contractcode,it’svitalthattheDAOpresentsthisinformation in the transparent, publicly accessible document. The absence of such transparency may erode members’ trust, thereby discouraging their active participation in DAO governance. Approach. We employ ChatGPT [53] as a question- answering system to determine whether the above 6 rules are present in the DAO documentation. Based on recent studies [54][55][56][57], ChatGPT surpasses existing Large Language Model (LLM) models in terms of performance in suchquestion-answeringtask.Furthermore,itexhibitssuperior robustness in question comprehension when compared with state-of-art question-answering systems. Querying LLM with a single complex question can lead to incorrect responses [58]. Related study [29] suggests that the Chain of Thought (CoT) reasoning method can enhance the LLM’scomprehensionofcomplexquestions.Inparticular,we break down the rules for the documentation into a series of Fig. 7. An example of querying ChatGPT with a chain of questions to intermediate questions. As shown in Figure 7, to check Rule check the Rule1 Member Participation is mentioned in the documentation. 1 (i.e., Member Participation), we raise three questions: Does Weremovethecross-verifyqueryforclarity. the DAO support governance?, Who can become a member of DAO?, and Can members participate in governance?. If all three questions are confirmed in the documentation, still operational. We think that the cause might be the lack we determine that the rule is satisfied. For all 6 rules, we of maintaining a website, leading only to popular DAOs can addressthemwithaseriesofquestionsbasedontheirdetailed create and sustain their websites. To validate our hypothesis, requirements from the DAO Model Law and merge similar we examine the DAOs with a TVL exceeding 20 million queries to form a question chain, as depicted in Figure 6. dollarsfromCoinMarketCap[3].Wefindallofthese11DAOs We utilize the ChatGPT model gpt-3.5-turbo-16k-0613 for still maintain their websites. We subsequently analyzed if the classification. The prompt for each query is shown in the low online website rate is due to DAOs being out-of-service. first box of Figure 7. We cross-verify the results with the ADAOisconsideredout-of-serviceifithasnotproposedany followingquery:Yourtaskistocheckifthesentencecontentis proposal within a year. We find that most DAOs (14,335 out mentioned in the document. Here is the sentence: [REASON]. of 16,427) are still active. However, only 183 of them have YouranswerformatshouldbeResult:Yes/No.Thedocumentis maintained their websites. providedbelow:[DOCUMENT].Ifthedocumentationexceeds To evaluate the effectiveness of the ChatGPT, we randomly the token limitation, we partition it into segments, each con- select 100 documentation and manually analyze whether each sistingof12,000tokens.Everytwosegmentsshareanoverlap documentation satisfied each rule. The results are presented of 2,000 tokens. in Table III. The results demonstrate that while the question Result. The number of collected websites and documentation chain can assist ChatGPT in accurately determining whether areshowninTableI.WefindthatforalltypesofDAOs,only a certain rule is mentioned in the documentation, the length a small proportion of them, specifically 622 out of 16,427, of the question chain can decrease the recall rate due to the provide their website. However, we discover that more than false negatives from ChatGPT. In addition to ChatGPT, we 460ofthesewebsitesareeitherofflineorhaveexpireddomain alsoevaluatedanotherLLM,Claude[59],forcomparison.The names. Consequently, only 183 of these DAO websites are evaluation, detailed in Table III, reveals that Claude can pro- 9 TABLEIV TABLEV INTEGRITYOFDOCUMENTATIONFROMCHATGPT. RESULTOFTHEIMMUTABILITYOFTARGETADDRESSWITHINPROPOSAL CODE. DAOType Rule1 Rule2 Rule3 Rule4 Rule5 Rule6 DAOType Open-source Close-source ByCREATE2 CanSELFDESTRUCT XDAO 1 0 0 0 2 0 Aragon 12 0 11 0 8 3 XDAO 96,588 568 14 0 Aragon 14,843 3 0 0 Tally 34 0 26 0 28 6 Tally 4,468 540 124 0 DAOhaus 1 0 0 1 0 0 DAOhaus 1,704 56 2 0 DAOstack 1 0 1 0 1 0 D Se A lf O -d s e ta v c e k loped 1, 2 1 9 6 0 1 0 5 0 0 0 0 Self-developed 3 0 3 0 3 2 Total 119,054 1,172 140 0 Total 52 0 41 1 42 11 A. Immutability of Proposal Code duceresultswitharecallratethatmatchesorevenexceedsthat Toassesstheimmutabilityoftheproposalcode,weanalyze ofChatGPTforthesequestions.ThissuggeststhatbothLLMs the target address within the proposal. The target address can provide sufficiently accurate outcomes. However, Claude refers to the contract to be called in the proposal code. It produces more false positives than ChatGPT. The higher rate should be open-source so that members can examine the code of false positives could be attributed to differences in training logicwithinthetargetaddress.Apartfrombeingopen-source, data or the possibility that Claude requires a different prompt the code logic in the target address must also be immutable. structurecomparedtoChatGPT.Thus,wechooseChatGPTto As discussed in Section V-C, the EVM opcode CREATE2 measure the integrity of DAO documentation. allows arbitrary change to the code logic inside the target The results of each DAO’s documentation and how they address while maintaining the same address. If the proposal alignwiththerulessetbytheDAOModelLawareillustrated codelacksimmutability,anattackercanarbitrarilymodifythe inTableIV.Ourfindingsshowthatnoneofthedocumentation code even after the proposal has been approved. For instance, from these DAOs adheres to all 6 rules. We discover that in the Tornado Cash Governance Attack [13], the attacker Rule2 Member Exit is not mentioned in any of the DAO initially proposes a benign proposal, which is subsequently documentation. Further analysis of the DAO Model Law approved.However,beforeexecutingtheproposal,theattacker indicates that this rule acts more as a compliance standard modifies the code within the proposal to transfer all assets. than a practical guideline for DAOs. In practice, the removal Approach.Toassesswhetherthetargetaddressintheproposal ofallDAOtokensbelongingtoamemberisconsideredasthe code is open-source, we follow the approach used in the member’sexitfromtheDAObydefault.AsforRule3Minority previous study [60]. We use APIs provided by blockchain Protection,onlytheDAObelongstoDAOhaus,hasmentioned scanners to check if the source code has been verified. We it in the documentation. After analyzing the DAOhaus [6] use the same method used in Section V-C to check the target platform, we find it incorporates the rage quit procedure into addressisunderthethreatoftheopcodeCREATE2.Weskip theirgovernancetoensuretheprotectionofmemberswithless the target address that belongs to the governance contract, as voting power. it has been evaluated in Section V-C. Answer to RQ2: Of the 16,427 DAOs analyzed, only Result. The results of the immutability of the proposal code 622 provide their websites. Among these, only 183 DAOs are shown in Table V. We discover that more than 99% continue to keep their websites operational, and none of of the target address in the proposal code are open-source. these 183 DAOs provide sufficient documentation for the This suggests that the majority of proposals maintain the governance process. clarity of their proposal codes. Among the 1,172 closed- source contracts, we identify 24 addresses that have been used by members, as indicated by more than 500 transactions associated with these specific addresses. This implies that VII. DOEXISTINGPROPOSALSENSURECONSISTENCY some members place their trust in these contracts despite BETWEENDESCRIPTIONSANDCODE?(RQ3) the noticeable lack of transparency. Regarding CREATE2, The governance process has become the target for attack- although we do find some target addresses created in the ers because the code within the proposal is controlled by CREATE2chains,theycannotdestructthemselvesandthus members. As such, attackers can hide malicious code into are not at risk of being mutated. However, the attacker can the proposal with the intention of either gaining control over inserttheSELFDESTRUCT orDELEGATECALLinto the DAO or transferring all the assets. In recent years, there thetargetaddress’scodetomakethispotentialthreatfeasible. have been numerous instances of governance attacks within DAOs, resulting in the loss of millions dollars [9], [10], B. Consistency between Description and Code [11], [12], [13], [14], [15], [16], [17], [18]. To explore the securityissuesintheproposal,wefirstverifytheimmutability The proposal description should align with the proposal of the proposal code by ensuring the target address in the code, ensuring members are fully informed about the pro- proposal is open-source and is not created with the opcode posal’s intent and allowing them to cast an informed vote. CREATE2. Second, we check the consistency between the If there’s a discrepancy in consistency, it could jeopardize proposaldescriptionandcodebyverifyingallcodewithinthe the correctness of the final voting result. To verify the proposal is mentioned in the description. consistency between the proposal description and code, we 10 Fig. 8. Example of extracting the description intention from the proposal description. employaframeworkcontainingthreecomponents:description intention extractor, code action extractor, and inconsistency Fig.9. Theexampleillustratingtheextractionandenhancementofproposal codeintocodeaction. detector. Within the description intention extractor, we extract the description intention—identified as (action, target object, parameter)—fromtheproposaldescription,whichoutlinesthe (PoS) tags to each token within the sentence. The action is functions to be called or not called in the proposal code. The identified by the token that is labeled as Root in the PoS code action extractor is used to gather the code action (as tag. Its lemma either exists in our verb list1, or it aligns with detailed in Table VI) from the proposal code, which shows synonyms of words within our verb list, as determined by the the actual functions to be executed. Finally, we assess the synonyms database [64]. Additionally, the token that has a consistency between the description intention and the code direct object (dobj) relationship with the Root token is also action in the inconsistency detector. identified in the action. The target object is identified by 1) Description Intention Extractor: The description inten- tokens that have a compound relationship with the action tionisrepresentedasatuple(action,targetobject,parameter). tokens. Lastly, the parameter is identified by the rest tokens Theactionreferstothefunctionnametobeperformedbythe with PoS tags such as NOUN, NUM, PROPN, or X. proposal code (e.g., transfer, update, approve), target object As shown in Figure 8, the action is highlighted in the blue is the target of the function call, and parameter denotes the box, the target object in the yellow box, and the parameter detailed parameters used by the action. We adopt a two-step in the green box. We also identify whether the description process to extract the description intention from the proposal intention originates from negative or positive sentences. To description. First, We identify all the code-related sentences identify these negative sentences, we utilize the BERT to that describe the function calls in the proposal code. After determine whether the code-related sentence is positive or that, we extract the description intention from these code- negative. When extracting from these negative sentences, we related sentences based on their grammatical structures. The assign a Negative tag to the description intention. example procedure of the description intention extractor is 2) Code Action Extractor: The code action extractor’s shown in Figure 8. The sentence in the red box is identified purpose is to extract the proposal code and enrich its content, as code-related. Subsequently, during the intention extraction, resulting in the code action as illustrated in Table VI. Given the code-related sentence undergoes parsing to form the cor- that the proposal code is in bytecode format, verifying its responding semantic dependency parse tree. The description consistencywiththedescriptionintentioncouldresultinfalse intention is then extracted based on the part-of-speech tags negatives. For instance, in Figure 9, the proposal description and syntactic dependencies in the parse tree. outlinesitsobjectastransferofARENAtokens.Itischalleng- Sentence identification. We apply the NLTK [61] to split ing to determine if the code matches the description directly the proposal description into individual sentences. In order to from the bytecode. To address this, we transform the proposal identifycode-relatedsentencesthatdescribethecode,weusea code into code action to add natural language information. fine-tunedBERT[62]forthesentenceclassificationtask.Due Since the proposal code only contains the target address, to the absence of a dataset for code-related sentences in DAO value, function signature, and calldata from the proposal proposals, we created a dataset comprising 2,200 sentences code, the rest part of code action needs to be enhanced based randomlyextractedfromproposals.Weselect2,000sentences on these data. The target address symbol can be determined fromthisdatasettofine-tunetheBERTmodel.Theremaining by checking the contract address in the public name tag or 200sentencesareusedtoevaluatetheperformanceofthefind- the function named symbol() in the contract. To determine the tuned BERT. We manually annotate each sentence to indicate functionname,weattempttofinditeitherfromthecontract whether it describes the proposal code. The evaluation of ABI[65]ofthetargetaddressorfromtheEthereumSignature sentence identification is shown in Section VII-B4. Database [66]. We obtain the contract ABI from the source Intentionextraction.Toextractthedescriptionintentionfrom codeofthetargetaddressviablockchainscanner.Ifthetarget code-related sentences, we first use Spacy [63] to generate a syntactic dependency parse tree and assign part-of-speech 1https://drive.google.com/file/d/1I1mPkZMohjC8vINL9JvJSoN8SoymDTRO

11 TABLEVI TABLEVII COMPONENTOFcodeaction. RESULTOFCONSISTENCYBETWEENDESCRIPTIONANDCODE.THE descriptionintentionISSHORTFORDIANDcodeactionISSHORTFORCA. Name Explanation ConsistencyType Aragon Tally DAOhaus DAOstack Self-developed Total Target Address Contract address to be called Target Address symbol Contract address name in natural language Normal 776 977 24 684 75 2,536 LackofDI 14,601 743 674 30 449 16,497 Value Value to transfer LackofCA 680 632 0 1,296 43 2,651 Function Signature 4 bytes of function ID Incomplete 119 2,087 1,014 170 440 3,830 Function Name Function name in natural language –Function 37 1,848 894 110 276 3,165 –Parameter 215 9,186 1,644 356 544 11,945 Function Parameter Parameter of the function Incorrect 0 0 0 0 0 0 Proposal 16,149 3,953 1,712 2,190 1,047 25,051 Function 14,693 6,498 2,457 210 1,361 25,219 Parameter 32,162 12,489 1,981 533 816 47,981 address is closed-source, we turn to the Ethereum Signature Database [66]—the largest database mapping function signa- turebacktofunctionname—forqueryingthefunctionname. thesamemethodusedintheincompletefunctiontoverifythe Once the function name is extracted, we can decode the function is mentioned in the description intention. calldata into function parameter, due to the function name 4) Evaluation ofSentence Identification: The evaluationof containing the sequence and types of each parameter. If we fine-turnedBERTdemonstratethatitachievestheprecisionof are unable to locate the corresponding information, we will 0.97, recall of 0.85. False positives occur when sentences aim leave it empty. to explain the functionality of the proposal code, rather than 3) Inconsistency Detector: We determine the following 5 describing the actual proposal code itself. False negatives are types of inconsistency between the description intention and generated when sentences use only abstract nouns to describe code action. the proposal code. Lackofdescriptionintention.Weextractthecodeactionfrom 5) Evaluation of Incomplete Function and Parameter: the proposal code, but can not find any description intention. To evaluate the performance of incomplete function and pa- Lackofcodeaction.Weextractthedescriptionintentionfrom rameter, we randomly select 1,500 functions, which contain the proposal description but fail to find any code action. 3,122 parameters. We manually label each of the functions or Incompletefunction.Thistypeariseswhenafunctionpresent parameters as incomplete. The incomplete function achieved inthecodeactionisnotreferencedinthedescriptionintention. a precision of 0.81 and a recall of 0.87. The false positive Our analysis involves verifying whether every Target Address is generated because the BERT can not correctly handle SymbolandFunctionNamefromthecodeactionareexplicitly the relationship between some words with similar semantic mentioned in the corresponding Target object and Action meanings, especially for the description having only a few within the description intention. We compare the semantic nounwords.Thefalsenegativeisduetothedifferentmeanings similarity between the combination of function name and of the words between the real world and blockchain network. target address symbol from the code action with the action For the incomplete parameter, we achieve a precision of 0.83 in the description intention. If the semantic similarity score and a recall of 0.88. The false positives are caused when exceeds the threshold, we believe the function is mentioned the proposal description uses URLs to outline its function in the description since the description describes the function parameters. The false negatives are caused by the misleading in a similar semantic meaning. We utilize the Sentence-BERT name of the parameter. model[67]toconvertthecorrespondingsentencesintovectors 6) Result of Inconsistency between Proposal Description and calculate the cosine similarity between these vectors. The and Code: The results of the 5 types of inconsistency are threshold value is set at 0.75, following the official Sentence- showninTableVII.WeexcludeproposalsfromXDAO,asthis BERT examples [68]. platform does not support proposals that include descriptions. Incompleteparameter.WeverifythateveryFunctionParam- Instead, it only requires members to submit code directly to eter listed in the code action is referenced in the Parameter the governance contract as a proposal, which could interfere section of the description intention. We assess parameters with the analysis results. Our results suggest that members of types: address, number, and byte, as well as their corre- currently do not pay sufficient attention to proposals. Of the sponding lists. For the address type, we first retrieve its name 25,051 proposals analyzed, 19,148 either lack a description using the method described in information enhancement. If of the proposal code or only contain a description without we can extract the address name, we verify whether the name corresponding code. Furthermore, among the 5,903 proposals appearsintheparameterfromthedescriptionintention.Ifwe that do include both a description and code, 3,830 are found cannot find the name, we directly check whether the address, to be incomplete, either lacking an explanation about the in hexadecimal format, is mentioned. For the number type, functionsordetailedparametersinthefunctions.Thisexplains we verify whether the number is shown in the description why attackers frequently target proposals in the governance intention. If the target contract is an ERC-20, we divide the process, as approximately 90 percent of proposals fail to value by its decimals [69]. For the byte type, since it can be a provide a clear and consistent description of the code for the hexadecimalrepresentationoftext,wecheckboththeoriginal members. content and the decoded text. 7) Real-World Attack Cases: To assess whether our ap- Incorrect proposal. The function in the code action is men- proach is capable of detecting real-world malicious proposals, tionedbyadescriptionintentiontaggedasNegative.Weapply we have gathered reports of DAO governance attack cases 12 TABLEVIII Ethereum [1] and the DAO Model Law [30], DAOs must be CLASSIFICATIONOFTHECOLLECTEDREAL-WORLDGOVERNANCE governed by smart contracts. Therefore, off-chain governance ATTACKINCIDENTS.THEdescriptionintentionISSHORTFORDIANDcode DAOs fall outside of our scope. actionISSHORTFORCA. Incidents Date Result ExpectLost ProposalConsistency IX. IMPLICATIONSANDSUGGESTIONS. TrueSeigniorageDollar[14] Mar2021 Successed $16K LackofDI Yuan[17] Sep2021 Successed $250K LackofDI Based on our research findings, we recommend that DAO Venus[74] Sep2021 Successed $250K LackofDI BuildFinance[16] Feb2022 Successed $470K LackofDI platforms should ensure that all DAOs established on their FortressProtocol[12] May2022 Successed $3M IncompleteParameter Beanstalk[18] Apr2022 Successed $182M IncompleteFunction platformsadheretotheprinciplesofdecentralizedgovernance Audius[11] Jul2022 Successed $1.1M LackofDI YAM[9] Jul2022 Blocked $2.1M IncompleteFunction ratherthanpermittingdeveloperstooptionallysupportit.With SwerveFinance[75] Mar2023 Successed $1.3M LackofDI TornadoCash[13] May2023 Successed $2M CodeMutability regards to developers, they should be obliged to disclose AtlantisLoans[76] Jun2023 Successed $1M LackofDI BIGCAP[78] Sep2023 Blocked $45K IncompleteFunction all privileged addresses to their members or, alternatively, IndexedFinance[77] Nov2023 Blocked $158K LackofDI mandate that all privileged functions be owned by the gov- ernance contract. Furthermore, they should provide complete from the following sources: Slowmist [70], CryptoSec [71], documentation to aid members in participating in the gover- Rekt [72], and Twitter [73]. We total collected 11 DAO nanceprocess.Blockchainscanners,suchasEtherscan,should governance attack cases [17], [14], [16], [18], [11], [74], label contracts that are deployed by the opcode CREATE2. [9], [12], [75], [13], [76], [77], [78]. Upon examining these In response to the observed inconsistencies in proposals, malicious proposals with our approach, we identified all 13 we suggest the DAO should force the consistency between proposals as 8 malicious proposals due to lack of description proposal description and tools and develop tools that can intention, 3 proposals due to incomplete function, 2 proposals automatically supplement proposal descriptions with missing with incomplete parameter, and 1 proposal is subjected to proposal codes and explanations. mutability of proposal code. X. RELATEDWORK Answer to RQ3: Of the existing 25,051 proposals, 22,515 of them (approximately 90%) fail to provide a consistent DAO. Recent research on DAO focuses on the DAO activity description and code for their members. Moreover, 16,497 analysis [20], [21], [7], [80], [81], [22], DAO definition and of these proposals do not provide any description of their application [82], [83], and DAO governance method [28], intended purpose. [84], [85]. However, they do not concentrate on the security aspects of DAO governance. As for empirical studies that do focus on security within DAO governance: Feichtinger et VIII. THREATTOVALIDITY al.[24] provided analysis on 21 on-chain governance DAOs, Limitations of complete DAO data. To counteract the threat specifically focusing on the voting process within the gover- of complete DAO data, we utilized several methods. First, nance procedure. Fritsch et al.[27] focused on the distribution we collect DAO data sources not only based on previous of voting power among three popular DAOs: Compound, studies [36], [26], but also from famous industry datasets, Uniswap, and ENS. Sharma et al.[25] analyzed the existing such as DeepDAO [8]. Second, we collect DAOs not only centralized risk of 10 existing DAOs and the corresponding from Ethereum but also from 8 different blockchains. Third, members voting behaviors. Wang et al.[26] analyzed the we collect DAO data from both websites and blockchains to design principles of DAOs from off-chain voting platform further make the dataset more complete. Hence, our dataset, Snapshot. Liu et al.[23] focused on voting behavior in DAO encompassing over 37,000 DAOs, 600 websites, and 100,000 governance. Dotan et al.[22] disclosed the centralized voting proposals, represents the most comprehensive DAO dataset to nature of four DAOs and explained the existing governance date. The results derived from this dataset can be considered attack incidents. The above research primarily focused on a representation of the entire DAO ecosystem. partial aspects such as voting within the DAO governance DAOs from non-EVM-compatible chains. According to framework, and their datasets are limited, no larger than statistics from DefiLlama [79], EVM-compatible chains cur- 1,000 DAOs. Our methodology analyzes the security issues rently dominate the blockchain. They account for over 85% across both on-chain and off-chain parts of the governance of the TVL across all blockchains. Therefore, we primarily framework. The security threats we studied have not been apply our approach to EVM-compatible chains. However, explored in previous research. apartfromtheimmutabilityofcontracts,ourmethodologyand Smart contracts analysis. Smart contracts have gained obtained insights do not exclusively rely on features specific popularity for facilitating trustless code execution on the toEVM.Therefore,ourapproachcanbeappliedtonon-EVM- blockchain. However, with the increasing usage of smart compatible chains as well. contracts, they have become targets for attacks. Numerous Off-chain governance DAOs. In off-chain governance, the tools have been developed for the analysis of smart contracts. governance process takes place on the website, where mem- Some notable examples include Mythril [86], Manticore [87], bers submit proposals and cast their votes. The execution of and Oyente [88]. Pied-Piper [48] proposed a hybrid analysis these proposals is carried out by the DAO developers rather method that combines datalog analysis and directed fuzzing than being automatically triggered by smart contracts [32], to detect potential backdoor threats in ERC token contracts [33]. According to the definition of DAOs provided by in order to enhance smart contract security. Beyond the direct 13 analysisofbytecode,binaryliftertoolssuchasGigahorse[89] [14] “True seigniorage dollar attack.” https://twitter.com/TrueSeigniorage/ transform the bytecode into a higher-level, function-based, status/1370956726489415683,2023. [15] “Pride punks dao attack.” https://twitter.com/BoringSecDAO/status/ three-address representation. Our method targets the detection 1556150989140373504,2023. of security issues within governance contracts and can be [16] “BuildFinancesuffersfromgovernanceattack.”https://cryptoslate.com/ integratedwithexistingtoolstoenhancethesecurityofdApps. build-finance-dao-hostile-takeover-treasury-drained/,2023. [17] “Yuan.finance attack report.” https://medium.com/yuan-finance/ Consistency between code and natural language descrip- yuan-governance-attack-update-and-migration-plan-3b5d949ab466, tion. The consistency between the code and natural language 2023. description has been well-studied [90], [91], [92], [93], [94], [18] “Beanstalk Exploit — A Simplified Post- [95]. They primarily concentrate on Java code and API docu- Mortem Analysis.” https://medium.com/coinmonks/ beanstalk-exploit-a-simplified-post-mortem-analysis-92e6cdb17ace, mentation, which are well-written and focused on describing 2023. code behavior. DocCon [96] detects inconsistencies between [19] “VPANDADAO RugPull.”https://twitter.com/DeDotFiSecurity/status/ documentation and the corresponding code for Solidity smart 1669859985113731082,2023. [20] Y.Faqir-Rhazoui,M.-J.Ariza-Garzo´n,J.Arroyo,andS.Hassan,“Effect contract libraries. Compared with Doccon, our method targets of the gas price surges on user activity in the daos of the ethereum different research questions. Our natural language description blockchain,” in Extended Abstracts of the 2021 CHI Conference on comes from proposal description, which lacks structured in- HumanFactorsinComputingSystems,2021. [21] Y. El Faqir, J. Arroyo, and S. Hassan, “An overview of decentralized formation such as tags in the comments or API document. autonomousorganizationsontheblockchain,”inProceedingsofthe16th Additionally, the proposal description encompasses a broader internationalsymposiumonopencollaboration,pp.1–8,2020. scope instead of only describing the code behavior. The code [22] M. Dotan, A. Yaish, H.-C. Yin, E. Tsytkin, and A. Zohar, “The vulnerable nature of decentralized governance in defi,” in Proceedings in our method is the bytecode, not the Solidity source code, ofthe2023WorkshoponDecentralizedFinanceandSecurity,2023. whichlackscodeinformationlikevariablename.Furthermore, [23] X. Liu, “The illusion of democracy? an empirical study of dao gover- our code size is extremely limited, containing only several nance and voting behavior,” An Empirical Study of DAO Governance andVotingBehavior(May8,2023),2023. bytesandthefunctioncallparametersratherthanthefullcode [24] R.Feichtinger,R.Fritsch,Y.Vonlanthen,andR.Wattenhofer,“Thehid- logic. denshortcomingsof(d)aos–anempiricalstudyofon-chaingovernance,” arXivpreprintarXiv:2302.12125,2023. XI. CONCLUSION [25] T. Sharma, Y. Kwon, K. Pongmala, H. Wang, A. Miller, D. Song, andY.Wang,“Unpackinghowdecentralizedautonomousorganizations In this paper, we conduct a comprehensive study of the (daos)workinpractice,”arXivpreprintarXiv:2304.09822,2023. issues in the DAO governance process components. We con- [26] Q.Wang,G.Yu,Y.Sai,C.Sun,L.D.Nguyen,S.Xu,andS.Chen,“An struct the dataset contains 16,427 DAOs, 183 documentation, empirical study on snapshot daos,” arXiv preprint arXiv:2211.15993, 2022. and122,307proposalsacross9differentblockchains.Thenwe [27] R. Fritsch, M. Mu¨ller, and R. Wattenhofer, “Analyzing voting power apply our novel methods to automatically identifying issues in decentralized governance: Who controls daos?,” arXiv preprint within these components. Our analysis of the governance arXiv:2204.01176,2022. [28] T. Dursun and B. B. U¨stu¨ndag˘, “A novel framework for policy based contract shows there are 665 DAOs the privileged functions on-chain governance of blockchain networks,” Information Processing within the governance contract can be controlled by unknown &Management,2021. entities that might be used by developers to intervene the [29] J.Wei,X.Wang,D.Schuurmans,M.Bosma,F.Xia,E.Chi,Q.V.Le, D.Zhou,etal.,“Chain-of-thoughtpromptingelicitsreasoninginlarge governanceprocess.Intermsofdocumentation,99%ofDAOs languagemodels,”AdvancesinNeuralInformationProcessingSystems, fail to provide documentation related to the governance pro- 2022. cess. Analysis of proposals reveals that 90% proposals fail to [30] “DAOModelLaw.”https://coala.global/daomodellaw/,2023. [31] “Compounddocuments.”https://docs.compound.finance/v2/,2023. provide a detailed description explaining the proposal code. [32] W.Reijers,I.Wuisman,M.Mannan,P.DeFilippi,C.Wray,V.Rae-Looi, A.CubillosVe´lez,andL.Orgad,“Nowthecoderunsitself:On-chain REFERENCES andoff-chaingovernanceofblockchaintechnologies,”Topoi,2021. [33] P. De Filippi and G. McMullen, Governance of blockchain systems: [1] V. Buterin et al., “A next-generation smart contract and decentralized GovernanceofandbyDistributedInfrastructure.PhDthesis,Blockchain applicationplatform,”whitepaper,2014. ResearchInstituteandCOALA,2018. [2] “Uniswap.”https://uniswap.org/,2023. [3] “CoinMarketCap.”https://coinmarketcap.com/view/dao/,2023. [34] “The Model Law.” https://uncitral.un.org/en/texts/arbitration/modellaw/ [4] “XDAO.”https://docs.xdao.app/,2023. commercial arbitration,2023. [5] “Aragon.”https://legacy-docs.aragon.org/aragon/readme,2023. [35] “Synthetify governance attack.” https://blockworks.co/news/ [6] “Daohaus.”https://daohaus.club/,2023. solana-exploit-dao-hacker,2024. [7] Y. Faqir-Rhazoui, J. Arroyo, and S. Hassan, “A comparative analysis [36] J.Arroyo,D.Davo´,E.Mart´ınez-Vicente,Y.Faqir-Rhazoui,andS.Has- of the platforms for decentralized autonomous organizations in the san, “Dao-analyzer: Exploring activity and participation in blockchain ethereum blockchain,” Journal of Internet Services and Applications, organizations,” in Companion Publication of the 2022 Conference on 2021. ComputerSupportedCooperativeWorkandSocialComputing,pp.193– [8] “Deepdao.”https://deepdao.io/organizations,2023. 196,2022. [9] “Yam attack analysis.” https://decrypt.co/104848/ [37] “Daostack.”https://daostack.io/,2023. yam-finance-safeguards-3-1m-treasury-governance-attack,2023. [38] “Tally.”https://www.tally.xyz/,2023. [10] “Potential curve dao attack.” https://gov.curve.fi/t/ [39] “Curve.”https://curve.fi/,2023. the-curve-emergency-dao-has-killed-the-usdm-gauge/2307,2023. [40] “MakerDAO.”https://makerdao.com/en/,2023. [11] “Audius dao attack.” https://cointelegraph.com/news/ [41] “Publicnametags.”https://info.etherscan.com/public-name-tags-labels/, hackerdrains-1-08m-from-audius-following-passing-of-malicious-proposal, 2023. 2023. [42] “Selenium.”https://www.selenium.dev/,2023. [12] “Fortressprotocolattack.”https://rekt.news/fortress-rekt/,2023. [43] “How to set up on-chain governance.” https://docs.openzeppelin.com/ [13] “Attacker hijacks Tornado Cash governance via contracts/4.x/governance,2023. malicious proposal.” https://cointelegraph.com/news/ [44] J. Xu, K. Paruch, S. Cousaert, and Y. Feng, “Sok: Decentralized attacker-hijacks-tornado-cash-governance-via-malicious-proposal, exchanges(dex)withautomatedmarketmaker(amm)protocols,”ACM 2024. ComputingSurveys,vol.55,no.11,pp.1–50,2023. 14 [45] L. Zhou, X. Xiong, J. Ernstberger, S. Chaliasos, Z. Wang, Y. Wang, [80] O.Rikken,M.Janssen,andZ.Kwee,“Theinsandoutsofdecentralized K.Qin,R.Wattenhofer,D.Song,andA.Gervais,“Sok:Decentralized autonomousorganizations(daos),”AvailableatSSRN3989559,2018. finance(defi)attacks,”in2023IEEESymposiumonSecurityandPrivacy [81] X. Zhao, P. Ai, F. Lai, X. Luo, and J. Benitez, “Task management in (SP),pp.2444–2461,IEEE,2023. decentralized autonomous organization,” Journal of Operations Man- [46] “Evmcfgbuilder.”https://github.com/crytic/evm cfg builder,2023. agement,2022. [47] M. Fro¨wis and R. Bo¨hme, “Detecting privileged parties on ethereum,” [82] E. Baninemeh, S. Farshidi, and S. Jansen, “A decision model for de- 2022. centralizedautonomousorganizationplatformselection:Threeindustry [48] F. Ma, M. Ren, L. Ouyang, Y. Chen, J. Zhu, T. Chen, Y. Zheng, casestudies,”arXivpreprintarXiv:2107.14093,2021. X. Dai, Y. Jiang, and J. Sun, “Pied-piper: Revealing the backdoor [83] L.Liu,S.Zhou,H.Huang,andZ.Zheng,“Fromtechnologytosociety: threatsinethereumerctokencontracts,”ACMTransactionsonSoftware An overview of blockchain-based dao,” IEEE Open Journal of the EngineeringandMethodology,2023. ComputerSociety,2021. [49] “Ethereum Constantinople/St. Petersburg Upgrade [84] C. Calcaterra, “On-chain governance of decentralized autonomous or- Announcement.” https://blog.ethereum.org/2019/02/22/ ganizations:Blockchainorganizationusingsemada,”AvailableatSSRN ethereum-constantinople-st-petersburg-upgrade-announcement,2023. 3188374,2018. [50] “Eip-1014: Skinny create2.” https://eips.ethereum.org/EIPS/eip-1014, [85] X. Fan, Q. Chai, and Z. Zhong, “Multav: A multi-chain token backed 2023. votingframeworkfordecentralizedblockchaingovernance,”inInterna- [51] M. Fro¨wis and R. Bo¨hme, “Not all code are create2 equal,” in 6th tionalConferenceonBlockchain,2020. WorkshoponTrustedSmartContracts(WTSC’22),2022. [86] B.Mueller,“Smashingethereumsmartcontractsforfunandrealprofit,” [52] “Tenderly.”https://tenderly.co/,2023. HITBSECCONFAmsterdam,2018. [53] “Chatgpt.”https://openai.com/blog/chatgpt,2023. [87] M.Mossberg,F.Manzano,E.Hennenfent,A.Groce,G.Grieco,J.Feist, [54] Y.Tan,D.Min,Y.Li,W.Li,N.Hu,Y.Chen,andG.Qi,“Evaluation T. Brunson, and A. Dinaburg, “Manticore: A user-friendly symbolic of chatgpt as a question answering system for answering complex execution framework for binaries and smart contracts,” in 2019 34th questions,”arXivpreprintarXiv:2303.07992,2023. IEEE/ACMInternationalConferenceonAutomatedSoftwareEngineer- [55] N.Bian,X.Han,L.Sun,H.Lin,Y.Lu,andB.He,“Chatgptisaknowl- ing(ASE),pp.1186–1189,IEEE,2019. edgeable but inexperienced solver: An investigation of commonsense [88] L.Luu,D.-H.Chu,H.Olickel,P.Saxena,andA.Hobor,“Makingsmart problem in large language models,” arXiv preprint arXiv:2303.16421, contractssmarter,”inProceedingsofthe2016ACMSIGSACconference 2023. oncomputerandcommunicationssecurity,2016. [56] Q.Zhong,L.Ding,J.Liu,B.Du,andD.Tao,“Canchatgptunderstand [89] N. Grech, L. Brent, B. Scholz, and Y. Smaragdakis, “Gigahorse: too?acomparativestudyonchatgptandfine-tunedbert,”arXivpreprint thorough, declarative decompilation of smart contracts,” in 2019 arXiv:2302.10198,2023. IEEE/ACM 41st International Conference on Software Engineering (ICSE),pp.1176–1186,IEEE,2019. [57] R. Omar, O. Mangukiya, P. Kalnis, and E. Mansour, “Chatgpt versus [90] L. Yu, X. Luo, J. Chen, H. Zhou, T. Zhang, H. Chang, and H. K. traditionalquestionansweringforknowledgegraphs:Currentstatusand future directions towards knowledge graph chatbots,” arXiv preprint Leung, “Ppchecker: Towards accessing the trustworthiness of android arXiv:2302.06466,2023. apps’ privacy policies,” IEEE Transactions on Software Engineering, 2018. [58] S.Zheng,J.Huang,andK.C.-C.Chang,“Whydoeschatgptfallshort [91] B.Andow,S.Y.Mahmud,W.Wang,J.Whitaker,W.Enck,B.Reaves, in answering questions faithfully?,” arXiv preprint arXiv:2304.10513, K.Singh,andT.Xie,“{PolicyLint}:investigatinginternalprivacypolicy 2023. contradictions on google play,” in 28th USENIX security symposium [59] “Claude.”https://claude.ai/,2023. (USENIXsecurity19),2019. [60] D.Das,P.Bose,N.Ruaro,C.Kruegel,andG.Vigna,“Understanding [92] D.Torre,S.Abualhaija,M.Sabetzadeh,L.Briand,K.Baetens,P.Goes, securityissuesinthenftecosystem,”inProceedingsofthe2022ACM andS.Forastier,“Anai-assistedapproachforcheckingthecompleteness SIGSACConferenceonComputerandCommunicationsSecurity,2022. of privacy policies against gdpr,” in 2020 IEEE 28th International [61] S. Bird, E. Klein, and E. Loper, Natural language processing with RequirementsEngineeringConference(RE),2020. Python: analyzing text with the natural language toolkit. ” O’Reilly [93] H.ZhongandZ.Su,“Detectingapidocumentationerrors,”inProceed- Media,Inc.”,2009. ings of the 2013 ACM SIGPLAN international conference on Object [62] J.Devlin,M.-W.Chang,K.Lee,andK.Toutanova,“Bert:Pre-training orientedprogrammingsystemslanguages&applications,2013. of deep bidirectional transformers for language understanding,” arXiv [94] Y. Zhou, R. Gu, T. Chen, Z. Huang, S. Panichella, and H. Gall, preprintarXiv:1810.04805,2018. “Analyzing apis documentation and code to detect directive defects,” [63] “Spacy.”https://github.com/explosion/spaCy,2023. in 2017 IEEE/ACM 39th International Conference on Software Engi- [64] “Synonym.”https://www.synonym.com/,2023. neering(ICSE),2017. [65] “Contract ABI Specification.” https://docs.soliditylang.org/en/v0.8.19/ [95] Y. Zhou, C. Wang, X. Yan, T. Chen, S. Panichella, and H. Gall, abi-spec.html,2023. “Automaticdetectionandrepairrecommendationofdirectivedefectsin [66] “EthereumSignatureDatabase.”https://www.4byte.directory/,2023. java api documentation,” IEEE Transactions on Software Engineering, [67] N.ReimersandI.Gurevych,“Sentence-bert:Sentenceembeddingsusing 2018. siamesebert-networks,”arXivpreprintarXiv:1908.10084,2019. [96] C. Zhu, Y. Liu, X. Wu, and Y. Li, “Identifying solidity smart contract [68] “Official sentence-bert examples.” https://github.com/UKPLab/ api documentation errors,” in Proceedings of the 37th IEEE/ACM sentence-transformers/blob/master/examples/app-lications/clustering/ InternationalConferenceonAutomatedSoftwareEngineering,2022. fast clustering.py#L57,2023. [69] “ERC-20TokenStandard.”https://eips.ethereum.org/EIPS/eip-20,2023. [70] “SlowMist.”https://www.slowmist.com/,2023. [71] “CryptoSec.”https://cryptosec.info/,2023. [72] “rekt.”https://rekt.news/,2023. [73] “Twitter.”https://twitter.com/home,2023. [74] “Venus protocol prevented hostile takeover attempt.” https://www. cryptotimes.io/venus-protocol-prevented-hostile-takeover-attempt/, 2023. [75] “Defunct swerve finance still subject of 1.3 million live governance hack.” https://www.theblock.co/post/222744/ defunct-swerve-finance-still-subject-of-1-3-million-live-governance-hack, 2023. [76] “Atlantis loans hack analysis.” https://blog.solidityscan.com/ atlantis-loans-hack-analysis-7f3fb2e295e0,2023. [77] “Indexed finance dao attack.” https://blockworks.co/news/ blackmail-thwarts-90k-dao-attack,2024. [78] “Bigcap dao attack.” https://twitter.com/BIGCAPProject/status/ 1697958233204490494,2024. [79] “Totalvaluelockedallchains.”https://defillama.com/chains,2023.