Is NFTX Down Right Now? Check if there is a current outage ongoing.

The subgraph has been reindexed. https://thegraph.com/explorer/subgraphs/Epp6gJotEJKAsJYdk7bDEURtGWWeZVXwZds2Pgw8eG51?view=Curators&chain=arbitrum-one Current version v1.4.0 QmYr6m1ESfvhKuZCQxyzy51o7oDWiwGgnHXL9MRZPuVvMr

Two transactions have been reviewed which has uncovered the issue. Transaction #1: https://etherscan.io/tx/0x395710b73d2156c87b11644d10db31b97f0a7a9ffd99c49ed6e988876d9adcab Occurred 5 days ago and was someone buying a PACA NFT from the V2 vaults through Reservoir. Transaction #2: https://etherscan.io/tx/0xa6bbc5d76880b29dae266ce31f9113f36741cb4e9d9888007a466285c1a384bb Occurred a few hours ago and was someone buying a PACA NFT from the V2 vaults through Reservoir. This transaction is the cause of the subgraph outage. The events on both transactions are the same (a few differences in the token holding values of the pool, but that would be expected based on price differences). Looking at the TokenId 559 (https://etherscan.io/token/0x3db5463a9e2d04334192c6f2dd4b72def4751a61?a=559) that was successfully bought in transaction #1, it ended up in the Vault through this transactions https://etherscan.io/tx/0xcd7bb8609e60aa2ac100ead2036eac47ed4982877003d9e4b63785e9ae1cdb2d which was a Safe Transfer (it was a sell into the NFTX vault). Looking at Token ID 9511 (https://etherscan.io/token/0x3db5463a9e2d04334192c6f2dd4b72def4751a61?a=9511) that failed/broke the subgraph, it arrived in the vault through a "Proxy Assert" 1000+ days ago. This is likely due to someone transferring two NFTs directly into the vault instead of going through the mint/sell process, which caused the missing `dateAdded` issue. (https://etherscan.io/tx/0x594a0d7dc671c854084295a0cad461a26e3377cc37a7d1aabde1d00a0e368d52) A fix has been deployed to remove the required value for the mint date, and the subgraph is reindexing now (est 4 hours).

The NFTX V2 subgraph has failed, we are working on a fix

The subgraph has now been upgraded to cover the issues with the fault indexing the transaction. All data is back and running again.

A sell evet on NFTX V3 has caused an error on the NFTX V3 vaults subgraph. Any transactions from block https://etherscan.io/block/20183594 hasn't been picked up yet. An update to the subgraph is in progress now, then will be pushed to studio and reindexed.

The indexer that pulls updates from the subgraph is not collecting the latest data. This is causing some buys/sells to not show up against the ongoing activity on V2 and V3 apps. The AMM reporting on NFTX V3 is showing the buys/sells against the pool, however the vaults/collections are not showing the new items in and out of the vault. This may cause some issues when trying to buy and item that no longer belongs in the vault, or selling the item and it not appearing immediately. This will not impact the protocol, all transactions will either fail or succeed, no NFTs or funds are at risk.

The subgraphs have been updated and user holdings are now appearing on the main sell page https://nftx.io/sell/ and individual vault sell pages.

On the sell page on https://nftx.io, as well as the sell pages on the individual vaults, the 721 and 1155 subgraph are not returning NFTs that are held by the connected user. We have identified the issue and are updating the subgraph. In the meantime, you can still sell your items into NFTX through https://explorer.reservoir.tools/ethereum and https://pro.opensea.io/

The issue was attributed to the Graph Index server. An abundance of requests through the graph led to some queries receiving 400 responses. Additional improvements have been made to the graph server and settings to improve caching and load capabilities.

The issue has resolved itself however we're still looking into the root cause.

There are currently 500 errros on a number of the NFTX V3 API requests. All vaults/funds/nfts are safu.

The API has been updated with new keys and is working again.

There is an issue with quote requests hitting the 0x API at the moment. We have identified the issue and are applying a fix.

The RPC node has been running successfully for the past 24 hours and we are closing this ticket.

A fix has been implemented and we are monitoring the results.

The RPC servers were returning a 503 response and casuing issues with both the API indexer and the app. These have been fixed and the site is working as expected.

We are still seeing buys and sells on the platform, and across other aggregators, however some actions like staking and unstaking are throwing errors on the app.

The RPC nodes for NFTX are behind the current block and some transactions may fail.

An update to the Inventory Staking contract has been pushed, which has fixed the temporary bug of new xTokens for existing vaults. Users who staked inventory positions in the past 24 hours will have received the wrong xToken. NFTX will provide a way to migrate these xToken across within the 7 days you will have been locked. Users who had previous staked positions can now unstake if you reach the end of your time lock. Users can now stake their inventory, which will be created on the orginal xToken address. Any fees earned during the 24 hours while the xToken address was changed will have been allocated to the DAO. The DAO will distribute these fees to the staked inventory providers based on a token-holding snapshot for each block where fees were generated. This will be actioned asap within the next two weeks. Any gas costs for the migration from the new xToken back to the old will be reimbursed. A full postmortem of the issue will be released to follow up on this status on https://blog.nftx.io

During a recent update to the Inventory Staking contract, a change in the formatting of the contract caused the byte code hash to change. This change meant that the hash of the byte code was different to the previous version. The hash of this byte code is used to programmatically determine the xToken address when staking NFTs into the pool, and since this was different, any new inventory staking events generated a new xToken address. The app has temporarily disabled inventory staking/unstaking while this is being fixed. Key points - Your NFTs and Staked positions are safe - We have disabled the ability to mint any staked positions on the app that differ from the OG while this is fixed - There have been two versions of some staked positions minted due to a bug, which *does not* impact collateralization nor the ability to redeem - There will be a review of the distribution of inventory fees to ensure that everyone receives their correct amount

Any new inventory staking have been created using a new xToken address following an update earlier today. While your positions are still the same as before, we are investigating if rewards will be distributed to these addresses and ensure that the marketplace and yield apps correctly show your positions. The apps are currently only pulling through the existing xToken addresses, so any new staked items in the past 12 hours will not be visible through the app.

We are experiencing issues with the front-end display of Inventory Staking positions at the moment. Investigations are ongoing, and updates will be provided as more details are discovered.

We have temporarily disabled the server-side rendering (SSR), which has fixed the issues we were having on the Arbitrum chain. This has a knock-on impact of disabling the social card shares, however it is preferential in this situation while we update the web app to us Rainbow Kit for wallet connections.

There is an issue with our the current implementation of UseDapp that we are using on the Marketplace, which Rainbow kit seems to be working fine. The Marketplace (https://arbitrum.nftx.io) is migrating to rainbow kit to alleviate the issues. Further details will be provided as updates are available

There is an issue with the network provider on the Arbitrum app at the moment; we are working on finding a solution now.

This incident has been resolved.

On the CoolCats vault, there was a swap 360 days ago for an item that the user didn't own. This was done directly on Etherscan (otherwise they couldn't do it on the UI), and instead of failing the contract emitted events that made it look like the id was added to the vault and then removed. As a result of this transaction one of the CoolCats was in the vault but not listed as one of the holdings. Earlier today someone swapped the CoolCat out of the vault which caused the subgraph indexing to fail — it was trying to put a date against an item that was removed that the subgraph thought wasn't in the vault, and NULL wasn't an accepted value against the date. To fix the issue we have made the `dateAdded` field on the subgraph an optional field, and the subgraph is working again. The new deployment is `QmeeGQ9xzXRf2vsYwi9MkuU8i4om6R9GUDhdp9q7j4oZyV`, and the decentralised graph and the hosted endpoint have both been updated.

The subgraph providing the data on the site has stopped indexing on block 16506629 after a swap transaction caused an error. We are looking into making an update to this to allow the syncing to resume. This is impacting the updated display for most areas on the site including - current holdings on vaults - new vaults - activity (buy/sell/swap) - staking/unstaking positions Any actions on the site (buying/selling/staking/swapping) will still work, but the results won't appear on the site until the bug is fixed on the subgraph.

This incident has been resolved.

The issue occurred when Hashmasks and Wizards had a free claime which resulted in the cycling of all the NFTs from the vaults. The cycle increased the difference in holdings on the vaults compared with the search holdings by more than 5% which triggered a manual check to ensure this was okay. Now the check is complete and the scripts are running as expected again.

We have noticed that some of the new items minted into the vaults are not showing up in the search filters. We are revieiwing this issue now and will update once we have identified this problem.

The GRT balance has been updated and queries are successfully being served.

There was unpaid GRT owing on the API keys that are maintained for Graph requests.

There is an issue with the graph responses on the NFTX marketplace and yield app. These are being looked into now.

With a bleeding-edge patch applied to one of our Erigon nodes the NFTX Graph Indexer was able to resync the NFTX subgraph. The site is now running as expected, and we will continue to closely monitor the service over the weekend.

There is an issue with the way in which Erigon is dealing with `trace_block` since the merge. This is resulting in a number of subgraphs that use trace to fail at the same block. We have tried to switch our indexer from Erigon to GETH to fix the issue, however GETH does not support trace so the subgraph still fails. Some updates have been made to the subgraph to remove the reliance on trace and a new subgraph has been deployed to test if this can fix the issue, the sync for this is looking to be 8-12 hours. It is now a race between Erigon pushing a fix for this we can patch our nodes and resume indexing, or once the new subgraph deployment is fully indexed and tested then we can update the endpoints. In the meantime we have reverted requests for the holdings to fallback to Web3 calls. This means that the vaults will load initially with the subgraph data before updating to show only items that are currenlty held in the vaults. NFTX will still not be able to show any recent activity or newly created vaults on the UI.

This appears to be an issue relating to both The Graph and Erigon. We applied https://github.com/ledgerwatch/erigon/commit/15ad43e1000e29e48f11de26aa42ebde72050772 patch however there were not improvements. There are some indexers which are able to index to the current block now so we are looking at whether those will begin to return the up-to-date data for the apps.

The NFTX subgraph (along with a number of other subgraphs) is failingin on block `15539509`. This is unrelated to the protocol being paused and likely linked to the recent merge. The Graph team is looking into the issue and we're awaiting further updates from them. You will still be able to interact with the app however data related to the holdings, recent transactions, activity etc may be incorrect (i.e. it will be showing the state of the platform as of block `15539509`).

This incident has been resolved. You can read more on https://blog.nftx.io/postmortem-nftxmarketplace0xzap-vulnerability/

The fix has been pushed and we're unpausing the contracts now.

Tests have all passed and the updates to the contact has been staged on Aragon. The next update will be when the votes are enacted in approximately 18 hours.

The fix has been written and is undergoing testing over the next few hours. Once these tests have passed and the updated contracts reviewed they will be staged on Aragon. Once enacted final tests on mainnet will be conducted and if successful the contract will be unpaused and protocol use will restart. Current estimated time for restarting the protocol is 36 hours from now (2100 UTC Thursday 15th September).

We have reviewed the report into the potential vulnerability on one of the periphery contracts and can confirm the vulnerability. Pausing the contracts means that there is no longer risk of exploitation. A fix for the issue is being worked on now and will be tested once completed. Once the tests have passed the updates will be staged and deployed and the protocol will be re-activated. Full postmortem will be annouced after the contracts are re-activated.

NFTX protocol is paused while we investigate a report into a potential vulnerability on a periphery contract.

The NGINX Proxy servers were getting errors on the response causing the response causing issues. The fix has been applied and the metadata is back.

The metadata API is currently experiencing some issues and images/metadata may not be loading. We are looking into the issue now.

This incident has been resolved.

The issue recetified itself, we are looking into what caused it.

There is currently issues with the NFTX Subgraph endpoint, we are looking into this now.

The search index has now completed and filters are back active on all vaults. To combat these recent issues we are now taking nightly backups of the search index to reduce reliance on the Algolia infrastructure.

The filters are currently unavailable on the vault pages for all mainnet collections. We have identified the issue with the search index and are reindexing now. These will be back in the next few hours.

This was an issue on the decentralised network and is back working again. https://status.thegraph.com/incidents/4zn6n5dtzbhd

We are currently seeing issues with the subraph endpoint and are working on a new deployment to fix these. Vaults may not appear with the latest details/holdings.

The issue has been resolved by Cloudflare. You can find out more about the issue by reviewing https://www.cloudflarestatus.com/incidents/xvs51y9qs9dj

Cloudflare is currently experiencing outages which will be impacting most of the internet this morning. The NFTX endpoings utilise Cloudflare which is causing the vaults/collections etc to not load. Once Cloudflare have corrected their issue the site will be back available.

The index has completed and all filters are working again as expected.

The records have dropped from our Algolia search provider. We are looking to restore the index from yesterdays backup while also rebuilding the index from scratch (this takes a few hours). We have also indentified that an error with the graph response caused the issue and will patch the script to avoid any false responses causing these issues.

The search filters on our collections are currently returning no options/results.

This incident has been resolved.

The new subgraph was deployed but the indexing wasn't up-to-date. This was fixed and re-indexed, everything working as expected again now.

Subgraph has stopped responding and we're not getting vaults and holding data back. We are looking into this now.

The API issue seems to be resolved now. We are looking into making Covalent our preferred metadata API provider while potentially building our own solution that indexes directly off the contracts.

OpenSea have experienced high levels of traffic and have disabled a number of API users. We have reached out to get our keys added back to the allow list and are still awaiting feedback from Opensea.

OpenSea are currently experiencing an outage on their site and API. The API issues are impacting the display of the images and some metadata around the items within the vaults. Where we already have cached the assets you can see them, but any not cached will not load at the moment. https://status.opensea.io/incidents/88862t31ybw5

While we were waiting for the shared hosting on The Graph to index we had a temporary infrasturcture which couldn't handle the additional traffic after the relaunch. This has been replaced with the graph index node that will be used for the decentralised graph index moving fowards. This has been operational for a few hours now with no further reported issues.

Some of the staking pages are not reflecting your permissions due to an issue with the requests to our subgraph. We are looking into the issue now.

The Graph is up and running again.

The issue has been identified and a fix is being implemented.

We are continuing to investigate this issue.

The SubGraph is currently experiencing issues. The site may not reflect the current holdings in the vaults while this is down as the app will be returning the last cached version.

Everything is working as expected.

thegraph.com looks to be coming back up and the vaults seem to be displaying, albeit a little slower than normal. We will continue to monitor the endpoint over the next hour.

The https://thegraph.com is down right now and impacting the loading of assets and vaults on the site. We are monitoring the issue and will follow up.

We are currently investigating this issue.

This incident has been resolved.

There was an issue with the production Graph endpoint which is now reindexing. We are using our backup endpoint while the reindex is working through to the latest block and we are seeing staked positions and new vaults through the front end. Everything is working as expected again, and once the product Graph has updated we will switch back across. There is no difference between the data on both versions, the difference is the production endpoint has a greater number of resources to fulfill large numbers of concurrent requests.

You may experience delay in the following areas - Vault Holdings - Viewing staked positions - New Vault creations The Graph is resynchronising now and will be available in a few hours.

Currently investigating and bringing up the backup graph.

The update to the new 721 subgraph has solved the issue, the app is working as expected again.

One of the endpoints on the Graph is failing to update and the fallback is using the cached response. We have change the endpoint and are testing the updates. These are showing the current set of NFTs from user wallets and we will be pushing the change shortly.

It appears SubGraph is current down. We are servining a cached version of the response however please be aware that some of the NFTs displayed in your wallet or in the vaults will not be up-to-date. This means that - items in your wallet may no longer be there if you've sold/transferred them - items in the vault may have already been redeemed - the vault may not show some items that have been added - your wallet may not show NFTs on the stake/sell page that you have.

The Graph is back up and synched with the latest block.

It appears SubGraph is current down. We are servining a cached version of the response however please be aware that some of the NFTs displayed in your wallet or in the vaults will not be up-to-date. This means that - items in your wallet may no longer be there if you've sold/transferred them - items in the vault may have already been redeemed - the vault may not show some items that have been added - your wallet may not show NFTs on the stake/sell page that you have.

The Kitty contract was written before the standards of ERC721 had been created, and thus does not have the same calls available on the contract that allowed the redemption of NFTs from the fund which work for typical ERC721. This limitation in earlier NFTs extends beyond Kitties and includes OG’s like CryptoPunks. This is why there are wrappers for CryptoPunks, it allows you to wrap a non standard NFT into a standardised contract which will then work across products built against specifications. In the future, NFTX may look to setup vaults for older projects that will only accept wrapped versions, but for now no official decision has been made. If you have any strong opinions, please mention it in a forum post or through the Discord.

Aragon vote has passed, contract is updated and the Kitty's can now be redeemed (this has been successfully tested).

The first patch covered off one problem but did not solve the redemption issue. Another patch has been put to vote at 0100 GMT 19th April 2021 and will conclude in 24 hours. We expect redemptions to be available for production in approximately 16 hours.

Aragon vote has passed, the fund is going through final testing.

Changes to the contract have been made and a proposal to make the updates is going through the 24hr Aragon voting process. You can find the updates on the discord at https://discord.com/channels/779073151115984926/791858971316453386/832656386755788839 Once passed the changes will be made, this will be around 6pm GMT 17 April. Once tested, the Kitty funds will be added back to the Gallery and App to allow browsing, minting, and redeeming and the warning will be removed from the site header.

The contracts are still being worked on to incorporate a fix for redeeming Kitties. All other funds are working as expected.

We are continuing to work on a fix for this issue.

@Gindi alerted an issue with Redeeming Kitties from the Gen-0 Vault at 03:29 GMT 15th April 2021. @javery replied at 07:34 and continued the conversation via DMs. It was identified at 17:00 that the NFTX Contract was calling the `safeTransferFrom()` function which is not supported by the Kitty contract (it is supported on 721). NFTX have hidden the Kitty Funds from the front end minting/redeeming and the Gallery while a patch is being worked on.

The Kitty contract was written before the standards of ERC721 had been created, and thus does not have the same calls available on the contract that allowed the mint request of NFTs from the fund which work for typical ERC721. This limitation in earlier NFTs extends beyond Kitties and includes OG’s like CryptoPunks. This is why there are wrappers for CryptoPunks, it allows you to wrap a non standard NFT into a standardised contract which will then work across products built against specifications. In the future, NFTX may look to setup vaults for older projects that will only accept wrapped versions, but for now no official decision has been made. If you have any strong opinions, please mention it in a forum post or through the Discord.

The request mint issue on the Kitty vault has now been resolved.

The vote has passed the request mint is once against available on the Kitty vaults. We will continue to montior the vault for any further issues.

Six more hours remaining on the vote https://client.aragon.org/#/nftx/0xf20e3d05813ce460d42994d26eb4b7d85381d117/vote/134/ , if successful requestMint will be available from 6am GMT.

There was an issue with the contract to allow for Request Mint. This was relying on a function that is usually present in ERC721 contracts however the Kitty contract isn't following all of the standards (similar to the Redeem issue with Kitties). A contract amends has been made and it will take 24 hours to go through Aragon before the fix will be released.

We are continuing to investigate this issue.

One of the community members notified us on the Discord channel that minting to the Kitty Gen-0 fund was returning "ALERT: Transaction Error. Exception thrown in contract code." on metamask. Currently Minting to either of the Kitty vaults is not available unless your Kitty already exists within the Eligibility list. The redeeming function is working still.

Subgraph has been updated and the Kitties pull through. It should be noted that if your Kitty is not part of the eligibility list you will need to check the "Show ineligible NFTs" to see them. Then, you can select them and Request to Mint.

The SubGraph has been created to cover this scenario and we are going through testing now.

We have reviewed the API requirements through the SubGraph and additional requirements have been specified. These changes require an overhaul of the existing SubGraph API's and will take two weeks to develop and test. During this time, minting on the new App UI is not available, however you can still mint via that https://nftx.org for both Kitty Funds. You can follow the docs ,https://docs.nftx.org/archive/old-tutorials/minting-fund-tokens, to see how to mint using the original interface. We will update this ticket once the SubGraph endpoint has been redeployed for monitoring.

The issue has been identified and is related to the Kitty contract not adhering to the 721 standards. As a result, these NFTs are not being picked up by our ERC721 SubGraph API. We are currently looking at updating the API to include non standard NFT contracts.

If you have Crypto Kitties in your wallet these are not showing as available for minting on the https://app.nftx.org

The approval process has been updated to work alongside the Kitty contract and minting through the new front end is once again avaialable.

Updates have been made to allow the approval of the kitties, however calling the standard approve method will result in an error on the Kitties contract. This is being worked on now.

The issue has been identified and a fix is being implemented.

The Kitty contract is returning a value which the new https://app.nftx.org/mint is treating as "approved", even when the assets have not been approved. As a result, trying to request mint or mint those kitties will throw an error. Kitties can still be minted on the old interface, https://nftx.org/#/fund/7.

The logic for showing NFTs that were eligible included tokenId >= range[0] && tokenId <= range[1]. This worked until another fix was added to cover assets with long ids which caused some eligible tokens to appear inelligible. This has been revereted using BigNumber methods and the bug is fixed.

Currently there is an issue with the SubGraph response which seems to be missing out some Deafbeef NFTs when checking users wallets. Any Deafbeef NFTs ranged between 0-127 are eligible for minting even if they are not being displayed on the https://app.nftx.org. You can still mint these Deafbeef Series 0 NFTs between 0-127 through our old interface https://classic.nftx.org/#/fund/67 We are looking into the SubGraph issue and will update the incident once we have identified the problem.

All Axie vaults are redeem only.

Axie Infinity have confired the contracts that allow you to breed, grow, and trade your Axies on the Ethereum network have been paused indefinitely. All further gaming for Axies will be done on the Ronin sidechain. The impact on the Axie Vaults mean that they are now only for Redeeming axies from the vault, nothing further can be minted.

Users have reported issues with Minting Axies into the Vault. On investigation we found that `"[FUNCTION]":"_transferFrom"` was erroring because of issues with the Marketplace contract. Eight days ago the MarketPlace contract was set to Paused by the Axies Devs (https://etherscan.io/address/0xf4ffe2ff12cafeab7034589d676c1de6af637484), and looking at their Discord this is due to a migration. Currently no Marketplace is operating and will be unavailable until this is unpaused. We are still investigating if there are any further issues which might be associated.

Contract has gone through aragon and the changes are now live, revokeMinting is now available for the Kitty contract.

The contract update has been made and is going through the Aragon process. Vote 135: Set new contract implementation (https://etherscan.io/address/0x0C68F11ad08EF6a02C4224fA6D6585D8b3102a26#code) to fix bug in revokeRequest function when handling cryptokitty vaults.

To have your Kitty approved to go into the vault a Request Mint is required. Once submitted, the team check to ensure that your Kitty is a Generation 0 for the Kitty-gen-0 vault, and that it is Generation 0 and Fast recovery on the Kitty-Gen-0-F vault. If the Kitty belongs, the request mint will be approved and you will be issued your token. If the Kitty is inelligible it will not be approved. The only way to get your Kitty back at that point is to call the Revoke Mint Request. At the moment this call against the Kitty contract is failing as the CryptoKitties contract does not follow the ERC721 standard contract. This issue has been found and a patch is being added. Once released this ticket will be updated to Monitoring and we a note added to discord.

### Incident Summary With our new NFTX v2 launch, we have decided to perform a gradual migration of the DAO treasury liquidity provided to PUNK/NFTX. In order to practice caution, we announced that the migration will be gradual and over time, starting with ~10% of the liquidity provided by the DAO. To begin the migration, custody of the liquidity was handed off to [Alex ](https://twitter.com/alexgausman)for a simpler migration. Alex approved 2 CryptoPunks for transfer to the vault contract, and immediately after sent the mint transaction to the vault to receive his PUNK tokens. However, this transaction was frontrun and ended up failing, with his 2 PUNK vault tokens going to the frontrunners address. Shortly after noticing something was off, the team's guardian permissions were used to pause all vault functions in order to give us time to investigate. After the attacker noticed our pause, they quickly sold the 2 PUNK tokens on the \(illiquid\) Sushiswap PUNK/NFTX pool, and made off with ~6 ETH. Soon after they were sold, we purchased back the 2 PUNK tokens for ~6 ETH, returning the tokens back to our custody. We have also staged an upgrade to solve the attack vector \(detailed below\), and will unpause the contracts after the upgrade is deployed. We did have a very successful audit with [Code 423n4](https://code423n4.com), however, the unique logic to handle these NFTs was added after the audit was complete, since we made the decision to handle bare CryptoPunks rather than Wrapped Punks later after the audit. We did go through an independent audit after our changes from Code 423n4, but it appears this flaw was missed due to the CryptoPunks contract not being within scope of the audit. ### Impact While the impact could have been 2 of the DAOs CryptoPunks \(or larger if we were not more careful\), after the attacker sold the PUNK tokens on Sushiswap, the team acted quickly to buy them back, reducing the total losses from ~33 ETH, to around **6 ETH**. No other vaults are affected with this issue, and nor were any other users affected. All vaults are currently paused, with an upgrade for the vaults already staged to solve the problem. ### Attack Vector Due to some NFTs \(CryptoPunks, Kitties\) not being aligned with the ERC721 standard, we have implemented some unique logic for them.  CryptoPunks, being rather early code before the ERC721 NFT standard, does not support normal approvals. They are performed by “listing” a CryptoPunk for sale to a specific address at a specific price \(0 ETH\). An approval for a contract to use a CryptoPunk is similar to “listing Punk #2550 for sale at 0 ETH to address nftx.eth”. However, even though the sale is approved for sale to a specific address, we did not verify that the transaction caller was the owner of the CryptoPunk, allowing anyone to execute that sale. So the moment Alex approved his CryptoPunks for “sale” and tried to mint, he was frontrun, with an attacker “spending” Alex’s Punks and receiving the 2 PUNK tokens instead of Alex. ### Solution  As a fix, we have staged an upgrade to the vault contracts that now verify that the person executing the order does indeed own the CryptoPunk. Before executing the buy, we check the CryptoPunks contract for who owns the punk ID being transferred, and make sure the sender of the transaction is indeed the owner of the CryptoPunk. ### Event Timeline **1:03:28 PM UTC**: Alex approves [2 CryptoPunks](https://etherscan.io/tx/0x29d893f55f5da4c3175c2bbd728b8f3f9ab8dd9217356065769a8905824a43b6) to be spent by the NFTX PUNK Vault. **1:05:04 PM UTC**: The attacker executes their [frontrun transaction](https://etherscan.io/tx/0x029b59bcf1264ca9396d952e6c4e04ad0a809d8de5ec4e7d10ce0fcc7dcca5dd) to “spend” Alexs CryptoPunks before his transaction to mint is processed and gains 2 PUNK vault tokens. **1:05:04 PM UTC**: Alex’s [transaction to mint](https://etherscan.io/tx/0xb885ac0c3f0dfc703f9b060b9b26f29fa12fdb6978a29a00e55e871a12858c71) fails, due to him no longer owning the CryptoPunks. We begin to investigate what could’ve happened. **1:22:06 PM UTC**: Since we purposely left the PUNK vault unfinalized for our soft V2 launch, we were [able to disable minting and redeeming](https://etherscan.io/tx/0x73b3f689fad146074841f08547ef6ddfcb9fbdb87135a19fcf0257dd4c3d52d7) for it. **1:29:03 PM UTC**: The attacker [sells their PUNK tokens](https://etherscan.io/tx/0xff4b93ceacbb8f93a7e177e89e94506ff07adce52a058488673449a908c1cdfa) into the rather illiquid Sushiswap PUNK/NFTX pool, selling both for ~6 ETH worth of NFTX. Later selling the NFTX through 1inch. **1:36:45 PM UTC**: We purchased back 2 PUNK \(6 ETH worth at the time\) of PUNK, retrieving back custody of the original stolen assets. **1:42:58 PM UTC**: We [execute a pause](https://etherscan.io/tx/0x231356d5a56e6ad4ae2689491851dba36c7747f3c4e516195ca806edebedb5d0) on the Vaults, using our safety Guardian roles. Just to be safe in case the issue is found elsewhere. **4:05:59 PM UTC**: [An updated contract](https://etherscan.io/address/0x20EA6c6c0F3d4405efC3E11466E314Fa7F4dB6A3#code) with the fix by Kiwi is staged to upgrade through the DAO, after which the vaults will be unpaused and NFTX v2 will be back to functioning as normal. ## Takeaway While we did go through a few audits and independent reviews, we made sure to be careful with our V2 launch as all new code should be used with immense care. Because of this, damages were reduced, and we were able to quickly isolate and solve the attack vector. Whenever we do support non-standard NFTs, we will be sure to audit the NFTs themselves as well, as their implementation may not be as clear as it seems. We hope this post mortem informs others to make sure native CryptoPunks and other non-standard NFTs are implemented very carefully. Thank you everyone for your patience and support.

The vaults have been unpaused and all functions are now available again on https://v2.nftx.org

As a fix, we have staged an upgrade to the vault contracts that now verify that the person executing the order does indeed own the CryptoPunk. Before executing the buy, we check the CryptoPunks contract for who owns the punk ID being transferred, and make sure the sender of the transaction is indeed the owner of the CryptoPunk. This fix has been added to the contracts and awaiting the Aragon voting process to have approval to enact the upgrades on contracts on mainnet.

Due to some NFTs (CryptoPunks, Kitties) not being aligned with the ERC721 standard, we have implemented some unique logic for them. CryptoPunks, being rather early code before the ERC721 NFT standard, does not support normal approvals. They are performed by “listing” a CryptoPunk for sale to a specific address at a specific price (0 ETH). An approval for a contract to use a CryptoPunk is similar to “listing Punk #2550 for sale at 0 ETH to address nftx.eth”. However, even though the sale is approved for sale to a specific address, anyone can execute that sale, so the moment Alex approved his CryptoPunks for “sale” and tried to mint, he was frontrun, with an attacker “spending” Alex’s Punks and receiving the 2 PUNK tokens instead of Alex.

An issue with receiving tokens back after minting has been discovered on V2 vaults. While this is being investigated the contacts have been paused. This affected the PUNK vault and DAO owned Punks, however we have retrieved the tokens already and all Punks are still owned by the DAO.

There was an issue with SubGraph and while trying to diagnose the problem resolved itself. Everything is back to working as expected.

We are continuing to investigate this issue.

There is currently an issue with the subgraph not syncing with the blockchain events. This means that the holding displayed on the vaults may not be current. After minting NFTs into the vaults it may appear as though they are not added, however they will leave your wallet and you will recieve tokens. When redeeming, you may be trying to redeem NFTs that have already been taken out of the vault by another redemption.

The updates to the contact have now been pushed and the Kitty redemption is working as expected.

The issue has been patched on the Kitty contract and the vote is on Aragon. We expect this to pass and enacted in the next 24 hours

There is an issue with redeeming Kitties from the V2 KITTY vault. The standard 721 uses `transferFrom` however the Kitty contract requeries the use of `transfer`. This is causing the preflight checks to prompt the wallet approval (metamask in most cases) to fail. You can still mint your Crypto Kitties into the V2 vault and receive your tokens, provide liquidity, and stake, however the redeeming is offline until the contract can be updated. Further details on the timing of the changes will be provided.

The fix has been pushed and all the vaults are returning the assests and images in recording time. Thank you for your patience.

OpenSea have banned all requests coming through CloudFlare workers. We updated the way in which we were retrieving the images however the load times were sub-optimal. We have set up another reverse proxy route into the OpenSea API and that has now been deployed.

We utilise the OpenSea API to provide metadata details about the NFTs in each of the vaults, providing data like name and image. At the moment the OpenSea API is heavily throttled and images are loading much slower than normal. The problem has been reported and we are awaiting further information.

When there are multiple vaults with the same contract we are only doing the lookup once. This is saving an additional 7 requests to the graph and is now showing the correct number of art blocks.

To show the available NFTs you have for minting we take all the current vault NFT contract addresses and use them to match against and NFTs you have in your wallet. Currently there are 8 Art Block vaults, but they all share the same Contract address (we separate what is eligible in those vaults through a range). We were then querying the wallet 8 times to find art blocks, and not making the array unique (hence why it showed that you had 8x the number of artblock NFTs available to mint.

We are seeing an issue where Art Block vaults are listing more NFTs available to mint than are in your wallet. When viewing the vault mint screen we are seeing multiple versions of the same NFT. A refresh of the page fixes this. This is a UI issue and doesn't impact the minting/redeeming. Further updates to come.

The graph is back up to sync again.

The fix has been applied and we are awaiting the subgraph to finishing syncing back to the current block.

Any new mints/redeems will will not show up on the current v2 front end. The last successful update was at midday today. We're investigating and working on a fix.

Updates have been made to the front end to ensure that you can still see your staked positions in these situations. Additional inline documentation/warnings in the app has also been added to alert users that when they are zapping to provide liquidity for an existing position they will be unable to claim fees or exit the previously approved position until the time lock on the current request has completed.

We have more details around the specific instance where this happens. If you… 1. Have existing Pool with unclaimed rewards 2. Zap into the same pool … then you may no longer see your staking positions. To reiterate, you still have your position and you will accumulate fees even while this visual bug exists.

If you have existing staked positions on NFTX V2 there is a bug where they are no longerbeing displayed if you Zap a new staked position. The locked positions will continue to show, and while your other positions are not shown they are still there and you will still earn fees. We are currently looking into the cause and will push a fix shortly.

The Graph is back up and the site is working again.

TheGraph is experiencing issues right now and our app is not loading any data. This will hopefully be resolved very shortly. We're in the process of moving to TheGraph's decentralised infra which will solve these types of outages

The endpoint is not longer returning 429 requests and we've logged a ticket with OpenSea to investigate further.

The Metadata API are receiving timeout requests to OpenSea so none of the images are loading for NFTs on the site. We are currently looking into the issue.

The graph is back up and running as of 19:27.

The Graph has had another outage. Going to look at a fix while we wait for them to come back online. unfortunately the issue stems across all subgraphs including ERC721 subgraphs, Sushiswap etc so hands are tied a bit. We are moving to new subgraph infrastructure soon but in the meantime we are at the mercy of The Graph's hosted service. They are currently working on a fix.

Mint and Staking is now back and functioning. You may experience some issues with Gas transaction estimations causing the process to fail, this is due to the pools on that particular vault not having been migrated across. If you do experience this, please contact us on Discord support https://discord.gg/pUxMFCu9km

The fix is going to be applied Thursday from 2pm UTC and will take a few hours. You can read more on https://blog.nftx.org/timeline-for-final-migration-and-re-launching-zaps/

Local testing has been completed and we are deploying the updates to Rinkeby for a final quite of testing before rolling it back into production.

We are now deploying a fix for a bug in the Mint & Stake zap contract that was discovered last week. The fix will require stakers to migrate to the new staking contract which will be accessible from the new UI. The staking page will be offline for the next hour while we carry out this work. We'll send another announcement when the UI is back up as well as further details on the staking migration.

We have identified a minor bug in the contracts used for the "Mint & Stake" function that stops users from claiming their rewards. All funds are safe and we expect to have a fix ready soon. Very sorry for the inconvenience. The only users affected are those who have used the "Mint & Stake" zap, and the effect is that their accrued rewards (up until we deploy a fix) will not be claimable on those vaults. The rewards are still safe, so we will be able to manually account for missed rewards and redistribute via the DAO in the coming weeks. Apologies again and we'll update as soon as we've made progress :pray:

There is a bug with the Staking Zap. We are currently investigating and will update shortly.

The graph has come back up again, and the site is returned to normal.

The issue has been identified and a fix is being implemented.

Subgraph is down so no vault data is being returned at the moment. We are looking into the issue.

A fix was pushed to fix this issue and it is now working as expected. Thanks to Alex Brandes, https://github.com/AlexBrandes, for finding a work around and adding it to the GitHub issues https://github.com/EthWorks/useDApp/issues/289#issuecomment-901546145

We have noticed that since the update to Metamask on the 18th August some users are experiencing the NFTX App crashing and showing a white screen with "Application error: a client-side exception has occurred (deeloper guidance)." Although you get this error, the TX is still going through in all cases so far and a refresh of the screen will bring the app back up again. We are investigating the issue and will look to push a fix for this tomorrow 20th August. Further updates to come tomorrow.

Sad Frogs are back! The DMCA order has been removed and the frogs are back on OpenSea, and the images are loading again on the site https://twitter.com/opensea/status/1429835505018023939?s=20

The Sad Frog's have been removed from OpenSea which is impacting our usual approach to displaying NFT images on the site. While some of the images are in our cache, any new items being minted are unlikely to appear. We are looking to bridge this issue by using the Covalent API to pull back Frog specific metadata.

The new endpoint is working and the vaults are coming back.

The graph endpoint had become stuck and needed a server reboot. This was done, and the endpoint has been switched back to the new version of the Graph which is more robust.

We are having connection problems to the Graph API endpoint, this is causing some issues when loading vault listings.

The site is working again. We will be introducing a fallback stale cache response for times when the Graph is unavailable. This will allow the vaults and NFTs to be dispalyed on the site, however they will be based on the last successful update and might mean that some target redeems are unavailable.

The Graph rectified their issues and the site is back up.

Vaults and NFTs are currently not pulling through due to an outage on the Graph. Further details will be provided as we learn more.