Brainless fudders stay out of this thread plz

brainless fudders stay out of this thread plz.


but seriously guys, what the fuck, lol

anyone (programmer/sa) actually had an issue like this in practice? it's not TOO hard to fanthom, how it could happen, and like all crypto software deals with unprecedented issues, but still, this doesn't really look normal lol

Other urls found in this thread:

etherscan.io/address/0x3f5ce5fbfe3e9af3971dd833d26ba9b5c936f0be
wiki.postgresql.org/wiki/Streaming_Replication
youtube.com/watch?v=AsRrQLY5s_8
twitter.com/SFWRedditVideos

brainless fudder here
AHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHHAHAHAHAHAHAHAHAHAHAHAHAHAHAHH exit scammed

I worked for a major bank as a big data expert and with a large scale hadoop file system. Transferring data between clusters can take a long ass time, but they should really have had a hot backup ready to go. Not sure what the issue is unless theyve had some hardware failure.

CHINKED

WHY DID I NOT FUCKING LISTEN WHY DID I LEAVE MY SHIT ON THIS CHINK EXCHANGE AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

I feel like this explanatory tweet is just going to confuse and piss off people

looks like theyre doing a really good job at keeping people updated

dude has a net worth over $2b, even after the crash

sure he'll become a fugitive and live undercover for the rest of his life so he can get a few more b

prove this fudder wrong!

You gonna be drive that Lamborghini soon, right?

The problem is theyre being pretty vague with what the exact issue was (which is reasonable), so people with experience in the field cant really gauge if we should be worried or not.

seems like the type of shit that would happen every fucking week back when i played world of warcraft and their maintenance took twice as long as it should have

fugitive? from who? the world police? crypto isnt regulated u mong. also he doesnt actually have 2 bil, hes going to have to slowly cash out

Programmer professionally since 07. Yes, these kinda things happen. Everything should be fine. Besides, you have your coins in your own wallets, right?

no (sane) person in the world has over a billion in cash, all billionaires would need to 'slowly cash out' if they wanted to convert all their net-worth to dollars

I'm a software developer.

It's really hard to know. It depends how the data is stored, etc.

There are so many variables, such as do they need to re-process data from a data warehouse to get it back into their processing system or is it literally just loading backups.

It's not unheard of for backup recovery to take this long, but they should have had a hot failover.

I'm still not sure what exactly happened, but I'd give it a day or two before I'd freak out.

I mean, they are probably freaking out over there, and started recovering from a backup, thought they could do it a quicker way and cancelled the original recovery, realized the second way doesn't work, and had to start over.

They also may need to fix infrastructure before they can even reload/process data, and that could take days depending on their deployment environment.

yea i know that. thats what i meant, a crypto billionaire doesnt mean shit. if anything he'll just pull an excuse like "oh we were hacked" so he wont look like the bad guy. scapegoat

I'm a lead distributed systems engineer at a top 100 internet company involved in AdTech. Having an outtage this major should be VERY rare, but given that it has happened and that they seemingly had no disaster recovery plan, it sounds normal. People in another thread mentioned the tweeting by the CEO sounds nervous, and it should be, because every dev during an outtage like this is shitting himself and probably pulling all nighters trying to fix the issue.


Also, blockchain is garbage. I know a lot of you probably never have ran your own blockchain node (aka wallet) and probably use exchanges, but you have to download the full blockchain (or a large % of it) before you can do anything. Mist has a known bug which prevents a majority of new users from synching the entire network, and thats what I'm guessing is causing the delay.

>fugitive? from who? the world police? crypto isnt regulated u mong.

you're a retard. all inernational law enforcement agencies including the FBI would be after him if he left with people's property. binance is even fully legal to use for americans

my dad works for binance, he said meet him in Bahamas for a looooong vacation?

No reason to panic when you can just check their address.

etherscan.io/address/0x3f5ce5fbfe3e9af3971dd833d26ba9b5c936f0be

Software engineer here. Databases can be very unwieldy beasts, especially one managing the transactions for something with as much SCALE as the #1 crypto exchange.

I'm pretty confident that things will be restored 100% accurately with minimal hiccups though. His team aren't amateurs.

>Besides, you have your coins in your own wallets, right?

and should he get caught how exactly will he get prosecuted? under which countries laws? ultimately it was you who decided to take the risk

question: if binance has shown that their data is fucked/fuckable, how can their records possibly be used by the IRS to enforce taxes? seems to me like iffy information means an easy legal battle for the traders

this is a fking joke.

>I know a lot of you probably never have ran your own blockchain node (aka wallet) and probably use exchanges, but you have to download the full blockchain (or a large % of it) before you can do anything.
This is what annoys me about this board. I've never heard of transaction fees until people were too lazy to download 20+ GB of data for owning their own wallet

I signed in 7 hours ago and my shitcoins were right where I left them. Let the FUD spread, I like cheap ETH

Transaction fees are baked into the protocol. You have to pay them even if you use your own wallet, although some blockchain protocols allow you to adjust the fee depending on how urgent you need your transaction verified.

Coder here.

Ummmm.....yes? Shit breaks?

Honestly though, I've noticed a lot of small UI glitches and general messiness on Binance. The problem is that testing can never demonstrate the absence of bugs, just their presence. Which is less usel than it sounds.

Meaning whatever pack of of pajeet built that shit undoubtedly fucked up somewhere. Gg Binance.

software developer (and part time server admin) here
they clearly didn't have a procedure in place on how to restore backed up data
if they did, they wouldn't be in this position having to try multiple methods to see what works best
you should already know that ahead of time
preferably you'd have a one-click solution to pull the backups from somewhere and run a restore script

My personal opinion.... after the accident they realized that their backup was fucked, and that they've lost their data.

They don't want the public to know about this, so they are desperately fighting to recover the data, or are trying t manually rebuild it based on the public ledger.

My guess is that they were depending on AWS auto-scaling (as it should have) but failed. This has happened for many Amazon customers.

Ive never configured a replica cluster, but if theyre literally restoring an image of master or a backup of the master, it could easily take DAYS if the thing is 100+TB. Even moving that data is slow, let alone writing it and rebuilding indexes etc.

Is this a good time to buy bnb?

>binance didn't backup their privkeys

Yeah they said it was a server issue which makes it much less likely that the data is actually corrupt and there was some service/hardware failure like you mention.

...

Why would he keep talking if this was an exit scam?

Also, hes already a billionaire.

Don't be stupid.

Hardware/infrastructure issues often result in data corruption. So it could be both, or neither

HEHEHEY
CARLOS MATOS HERE
BINANCEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE

Call me a fudder, but not in love with the idea that they decided to not go for method 3? I don't know shit about how this works but I'm assuming there's a good reason why it's slower.

may not be an exit scam but there might be a pretty big problem where you may never see your money again

AWS has a live triplicate policy (unless they have changed), but the backups are not stored on hot devices...sort of explains why it would take this long if primary failed.

Insider here. All three "methods" would fail. Sell everything you can now. You have 8 hours

I've actually built a system that could fail in a similar way way. There was a master database that was doing replication streaming to a bunch of other nodes that had a copy of all its data.

wiki.postgresql.org/wiki/Streaming_Replication

if the master fails, we'd need to rebuild the entire cluster and resync the data. but we had a redundant master so if it failed, we could switch to the other one. never happened, thankfully.

I wonder why they arent letting the sync finish. If it take 20+ hours to sync, I'd say let it run and just wait. I cant think of any faster options than that. I'm not sure how they can take something that should take 9 hours and make it take 1-2 hours. That doesnt make any sense to me.

yeah, just like how the fbi is all over Bittrex for stealing literally tens of millions from a small selection of about 10,000 accounts.

depends on the data
maybe they can split it up into 4 parts and run 4 smaller backups to 4 different databases at the same time, making it 4x faster

Method 1 is turning it off them back on again.
Method 2 is is finding where the fuck they put the HDD with the backup.
Method 3 is having CZ recreate the entire checksum on pen and paper via his photographic memory.

kek

Asian SWE from big data vendor. Obv I don't know what happened at Binance, but their explanation so far is not inconsistent with the problems you do have operating these systems in the real world.

Imagine Binance loses all their private keys and all the funds inside are permanently locked. Significant deflation + USDT getting wiped off the books. Could be awesome for the market as a whole.

Not to mention...

.. destroys confidence in all the other exchanges.
... used as excuse to regulate the entire industry.

>DEVOPS199 JUST DESTROYED THE MARKET SELL SELL SELL
In the long term the deflation is way, way more relevant

Yes, it’s totally normal. This is the largest exchange in existence. I guarantee their dB is in the high terabytes, and takes time to restore. Also, you can see their wallet addresses, which if you remember, is the whole point of crypto. Their coins haven’t gone anywhere

Not our problem, we pay or fees without delay and only ask for service.
Keep the fucking site online or stay out of business.
Fucking binance shills.

I’m not saying it’s not shitty business practice, but everyone saying they were hacked is retarded

Then don't use binance asshole

I'm an actual Technical support guy for enterprises/businesses.

Transferring terrabytes of data takes hours, if not days. Real things man.

Our HDFS is so big once when we had a failure we had to fly backup HDs from cold storage to our datacenter

Most newfags won’t remember this but one time Veeky Forums was down for a couple days because moot had to drive the backup discs from Texas to CA when he changed service providers.

If the post I saw earlier about them giving free ETH as compensation is true, I bet it comes at the cost of agreeing not to sue.

Their exchange, down for over a day. People can't get to their money during one of the most critical stock market days in years.

Quite the fuck up.

You expect more out of an unregulated industry where anonymous people trade a commodity with an indeterminate value.

>all these posts popping up suddenly
>developer here, everything will be just fine!

Based on what?

Unless you have access to their equipment, you're really in no position to know how serious the problem is

>People can't get to their money during one of the most critical stock market days in years.
See now you have to wonder. Is it really a coincidence? Right after the SEC shilled crypto?

my dad works at microsoft

This shit bout to get ugly niggas, once the binance paid shills come to defend their master means something is going up.

Do you think they could try to pose as hedge fund managers when they want to dump their bags?

Worked for APX, Sony and the UN as a software developer. This is a typical procedure just transfering data. Takes hours or days. It will be OK.

I'm still wondering about my original theory, which is this...

... they were trading on their own account. Borrowing customer coin to long or short the various currencies, and they got caught upside down with the price action earlier in the week.

More than the correlation to the stock market, I find it suspicious that this happened right after the BTC price went crazy with the below 6000 up above 7400 price action we saw for that one hour.

> If the post I saw earlier about them giving free ETH as compensation is true
Where did you see this?

i thought they are doing a system upgrade, why are they doing data recovery?

this is the dumbest shit I've ever seen. If option 1 is the fastest and he lists no downsides..why wouldn't they just use that option and not tweet about 2 other much longer options?

fucking jokers

That didn't happen here because Binance moved 30k BTC from their cold wallet to their hot wallet at 7pm.

shit happens
disclaimer: I only have ~20 BNB in Binance
I converted from USDT -> USD during the Tether FUD

> If the post I saw earlier about them giving free ETH as compensation is true,

This fucking guy

the real question is why arent these noobs using VEEAM

right? I literally logged in 10 times today, nothing is missing. only around 5PM EST it was a little slow but I was still able to get in

...

time for binance refugees to join

THE PEOPLES EXCHANGE

ku coin

networking specialist here. looks like someone fucked up majorly and probably taking ages to initiate their disaster recovery

So they have some BTC

Doesn't mean they'll give you any

LMAO

yeah, same. my shit is fine

still, going to move everything to a wallet. this is a good wake up call

In Chinese, "disaster recovery" translates to....

... move to the hills, never seen again, sorry about all that money you used to have, but I need it more for hookers and blow.

I use veeam

What if the IRS is behind all of this?
In order to get a record of everything.

Binance has infinate money. They should have their entire cluster synced to multiple instances at multiple DCs.
Infinite money = NEVER go down unless youre retarded or stingy on hosting.

There is definitely a major fuckup at play here. More than they let on for sure

"Cheng, here's the money. We'll dump on them when they least expect it. Now the exchange back to working order or else.

i remember a tweet of him saying one of his engs threw up when they saw how long it would take(and why i assume)

maybe he saw the data was donezo and then i could understand that reaction.

Do you know how hard it is to keep multiple clusters in sync with the transaction volume they have? Its VERY fucking hard. I had to build something at work that did something similar. What do you think the latency SLA is for their backups? Daily backups? Hourly? Lol if you think they use replicated writes cross-DC at their transaction volume

>Lol if you think they use replicated writes cross-DC at their transaction volume
Its definitely possible. The DCs would be in same region and have massive bandwidth between them.
Throwing money at it makes it possible.

They did, but the problem is supposedly that the sync to the backup cluster got fucked, so they had to offline the master db in order to do a full resync. Their database size could easily be in the multi-petabyte range. That said, their architecture should allow for multiple backup clusters, but most likely it's some bullshit like a single mysql slave or some shit that they've had in place since before they got big.

You do not know what you are talking about. Atomic writes cross-DC is not feasible at their scale due to noise on the write. You have to make writes atomic across all data centers which means your write has to block until all data centers confirm ack the write.. If any fail the write must be retransmitted. I'm not even going to dignify any of your noobish comments anymore.

Okay my jimmies are starting to get pretty rustled here what the fuck

of course you could batch it. It doesnt have to be synced 100% in real time.
Its enough to be able to fall back onto a 5-10 min old version in case of catastrophic failure of your main production system.

eating the loss on compensating users that missed out in those 10 minutes is smaller than going down for hours on end.

If you think with unlimited budget you cant create a setup thats almost impossible to down, you are a noob adminlet.

>only having one backup cluster

Who would GOX now and not 1 month ago?

What noise you fucking retard. Its very feasible

guys why didnt they use a distrusted ledger for this

so obvious

have you followed the gox court case? If you lose most of your user's coins, after courts, you will only have to pay the fiat value at the time you lost them.

Mt Gox filed for bankruptcy because they were insolvent. However, the coins they have now became worth several times their pre-hack networth. He gets to keep the difference.

If binance was going to gox (which nip law now has a precedent for) they wouldn't do it at ATH, they would wait for a big dip (which is a LOT easier to predict with access to unfiltered market data).

Shit you are right, I forgot about that loophole/precedent.

Well gentlemen and traps, it's been a pleasure to fud and shill with you, see you in the valhalla or wherever you go after going hero.

> In heaven all the candlesticks are green

youtube.com/watch?v=AsRrQLY5s_8

>I mean, they are probably freaking out over there, and started recovering from a backup, thought they could do it a quicker way and cancelled the original recovery, realized the second way doesn't work, and had to start over.
I work in IT (being intentionally vague here) and this kind of shit is super common.