Saturday, May 3, 2014

Attack against LocalBitcoins infrastructure 3.5.2014

LocalBitcoins received a very dangerous attack against the site infrastructure on Saturday 3.5.2014.
For now
  • All user data and Bitcoins are safe;
  • The site will be down for a while as the system is being rebuilt

Details

LocalBitcoins hosting provided received a request to restart the LocalBitcoins.com website server and give access to the server console (root) on Sat May 3 13:32:27. LocalBitcoins team did not initiate this request. For now, it looks like the request was made using spoofed email addresses and other weakness in the hosting provider support system.
  • LocalBitcoins team was alerted about the abnormal activity when the hosting provider restarted the server.
  • The attacker gained a root access to the server for ~40 minutes before the attacker was kicked out and the server shutdown.
  • All data on the website server is encrypted. Manual actions are needed to make this data readable, so the attacker could not gain access to the data even when having a server console access. 
It is very unlikely that the attacker gained access to any data;  LocalBitcoins is still performing full investigation on the matter.
  • Bitcoins in hot wallet and cold wallet are safe, as LocalBitcoins runs its bitcoind and wallets on a separate server.
  • LocalBitcoins team has started to rebuild the website server on fresh hardware.
LocalBitcoins team will make further announcements when the investigation proceeds and the site becomes available again.  We expect to spend at least 24 hours on this. LocalBitcoins team apologizes the issues the downtime may cause to the users.



35 comments:

  1. Yes, thanks. A reasonable encryption strategy is to be able to decrypt with a password of at least 12 (more like 15 - 20) characters for access that lasts a few minutes, or a much longer password (40 characters or more) for access that lasts an hour or two. If we assume that the backer grabbed a copy of enough information to be able to start an exhaustive search for the password, LBC knows how long we can expect his search to go on before he finds it. If his search is ever successful, what data will be compromised? Is there a list of BTC addresses that LBC can provide to miners, asking them to filter out transactions from them until further notice? A protocol for that kind of lock would be nice. I would honor it if I were mining. Just a simple request "please lock this BTC addy until further notice," signed with the address would do. It could be broadcast in any transaction and thereby get to all miners. I'm posting this comment to bitcointalk too.

    ReplyDelete
    Replies
    1. That's a great idea. Keeping the spirit of the protocol in mind.

      Delete
    2. We will highly probably reset all bitcoin addresses. The old private keys/addresses will still be stored, but using them is not recommended and we will only transfer funds from those manually, periodically.

      I don't think it is very likely that all miners, or even small minority of them would start blocking transactions, because it requires effort from them. And it would be useless unless a very big portion of the miners, 90% or so, would start using it.

      Also it is almost 100% certain that the attacker doesn't have any access to even to encrypted private keys. However there is a very small possibility that the attacker might have access to encrypted user data.

      Delete
    3. Will all the ongoing transaction be still on an safe? I am asking as I was just in the middle of one with fund being transferred but bit coins still on hold

      Delete
    4. Yeah, all transactions should be safe. The operations will operate exactly from the same state as they were when the server was shut down.

      Delete
    5. I have to disagree.After checking blockchain I can see my wallet has been systematically emptied over the last 2 days shortly after each deposit reached the account. .

      Delete
    6. I made a SEPA transfer and the bitcoins are still to come. Just wanted to make sure the transaction is still active and all the info will be available

      Delete
    7. Your comment hurt my head. Unless I misunderstand what you're saying, you're pretty much implying that disk encryption works by storing the password on the disk that an attacker can 'search for' in 10-20 minutes.

      Delete
  2. So your provider has root access and can give anyone console access remotely? That is not taking security seriously.

    ReplyDelete
    Replies
    1. Unless you have a colocated server with your own lock on the cabinet, it's common that the provider can grant remote access, reset root password or do reboots on request. Sometimes, unfortunately, to the wrong person. Without a colocation, you don't even have physical access to the server you rent, so obviously the provider needs to be able to physically control it. If you encrypt your data, it's not a huge issue. In fact I applaud localbitcoins for encrypting the data, because in the past similar attack vectors to bitcoin service providers were successful due to a lack of encryption (e.g. the linode attacks in Feb 2012, Bitcoinica, recent Canadian Bitcoins and plenty others that I do not recall at the moment).

      On linux, for example, you can use LUKS to encrypt to root file system and add the early-ssh initrd hook for manual unlocking in order to boot. If a server is compromised like in the situation above, after you regain control and kick off the attacker, you have the option of wiping the disk or reencrypting it (cryptsetup-reencrypt), so that even if someone manages to crack the password with a copy of the data he made, he can only acces the proporion of the data in that copy.

      Delete
    2. If it's a dedicated box, the hosting provider can only "grant root access" if they have the root password themselves, but it's unlikely.

      What happens in these attacks is the attacker gets some kind of KVM and then drops to maintenance mode or boots from the network in order to bypass the password entry.

      If the system is encrypted, they still can't do much, even like that.

      Bitcoinica, inputs.io and other infamous hacked BTC sites didn't even have their own dedicated server, just a VPS that you could access as root from the host machine that was under the control of the hosting people. With them, all you needed was to social engineer the hosting. With Localbitcoins, BitBargain and other sites who take this seriously, the box is dedicated and the hosting company can't get root, and the file system is encrypted.

      Delete
  3. when is the site going to be up again?

    ReplyDelete
  4. Hi,

    Are the coins safe? Please tell me that they are safe.
    When i checked blockchain, My wallet is emptied.
    https://blockchain.info/address/1P42x9KJgFkjano9WitAcbzTs5z12odfK7

    ReplyDelete
    Replies
    1. It's normal with shared wallets like Localbitcoins (and Bitstamp, and BTC-e, and BitBargain, etc).

      The address isn't dedicated to you for spending, it's just a way for Localbitcoins to know that the coins sent to that address should be added to your virtual balance. You're spending coins more or less random, from whatever's available. It's no problem as long as the virtual balances are correct and match the real coins.

      AFAIK only blockchain.info and coinbase.com transfer coins from the actual user deposit addresses.

      Delete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Okay. This is funny.
    Someone tried to attack the site.
    Its been a day or so. They said it was unsuccessful, nothing was stolen.
    WHY IS THE SITE STILL DOWN???
    All our money is frozen. I have lost over $850 because of the frozen funds.
    YOU ARE NOT DEALING WITH TOYS, but money. This is fucking serious, thats not a game where you say "oohh we are very sorry, we dont know when we will be back, but a very serious attack accured, but no problem bla bla"
    This is not something I would accept a simple apology for. You cant just go offline for days FUCK UP Peoples business. THATS STUPID. Situation getting suspicious. Watch out!!! GET BACK ONLINE ASAP!!!

    ReplyDelete
    Replies
    1. Not sure if troll or very stupid.

      Delete
    2. Just concerned bro, dont get it twisted.

      Delete
    3. "Concerned bro" is acceptable.

      Asking "WHY IS THE SITE STILL DOWN" in light of the clear message posted on their front page is just a waste of breath.

      Saying "YOU ARE NOT DEALING WITH TOYS" is implying they act like they are, when they're obviously doing the opposite by making sure everything is secure before coming back online.

      They did not say "we don't know when we will be back", they said it would likely be 24 hours.

      Their apology is not needed, because they didn't really make a mistake and they're actually doing what they should. You don't have much to accept or not.

      They do not FUCK UP people's businesses, it's people who risk their coins. If you pretend to agree to their terms but don't really know anything about them (such as the fact that they're not a multi-million dollar bank, but simply a website with no promises), it's not their fault, it's your fault. Bitcoin is a truly free market, if you know of a better site that gives you the same features, use them. Or create your own one and then take shit from commenters while you're losing a weekend thanks to other people's incompetence.


      Delete
  7. Agreed! They don't appear to be replying to comments on here, and I also asked for a contact email from them over twitter 5+ hours ago and have had no response!

    ReplyDelete
    Replies
    1. The best they could do was to get a monkey with a typewriter to respond to comments like these saying "We will be back soon hopefully, then it will work again". For a big enterprise, it's somewhat expected. For a site like Localbitcoins who have already posted the situation on their blog and their front page, it's redundant.

      Delete
  8. Calm down people and led them do their job without being cursed at or harrassed.

    1) A lot of us are losing money. That is the risk in BTC and if you can't handle it you shouldn't be dealing in BTC
    2) They said it would be 24 hours. It has only been 14 hours.
    3) I appreciate that they gave us a time frame at all and I expect it to take longer because rebuilding a server like that is a HORRENDOUS job
    4) I would rather them fix the problems than respond to questions from asshats on a blog
    5) Until they have it fixed and up and running it is unlikely that they will know the answers to many of your questions

    ReplyDelete
    Replies
    1. Anyone who deals in bitcoin already knows the risk of a fluctuating market value, this is not what is of concern.

      I cant speak for other but my concern comes from their statement sayin ALL wallets and coins are safe and were stored on a seperate server, but checking blockchain tells a different story.

      Losing money because of a drop in the market is one thing, but when coins start disappearing from wallets that are held on an encrypted server that the hacker supposedly didn't gain access to (especially after recent events on the bitcoin scene) people naturally start to become concerned.

      Delete
    2. 1) This has nothing to do with the nature of bitcoins.
      2) Its been 23 hours now, since they made the announcement.
      But the site has been down for more than a day.
      Get your facts right. When they coming back?
      3) I appreciate this short message they gave us for our coins, but rebuilding a server is not that big work, when they have everything stored. Why cant they just get the site going or something, while performing some test? If nothing really happened.

      Delete
    3. - They said it would likely take 24 hours. It's only been 23 as you say, so put the pitchforks down. They're going to come back when it's safe to do so.

      - You didn't build the system, you don't know if it's big work or not. Even if it's just a few hours of work, there could be other factors like just having started their sleep or physical stuff to do at the DC and needing to make an appointment, or anything else that you didn't think of.

      - Nothing didn't happen, root access happened. "Nothing" simply refers to no theft of coins or data. There's still a box that an attacker gained root to. If they left a backdoor somewhere and Localbitcoins did what you suggested, the coins would actually get stolen. Thankfully they know what they're doing and do not need to rely on your suggestion to simply restart the system that's been owned and 'perform tests' while the attackers steal everything.



      Delete
    4. So good to have you here Martin to explain the situation which you are probably very familiar with.
      Dont let them sleep untill this is fixed. Dont sleep on my money.
      Im not talking about pennys I am talking about thousands and thousands of my hard earned USD.
      If you'd know the nature of business I am in, you'd know that every second counts.
      Thankfully they know what they are doing as they just moved to a "Secure Swiss Server", which is working like a charm, as we can see.
      If the attacker whould have the chance to steal it in 40 minutes (very LONG TIME), he would have done so.
      Best believe me its over with for the hacker.

      Delete
    5. "Dont let them sleep untill this is fixed" ('until' btw)

      Once I'd been up for over 24 hours and just when I was about to pretty much faint into bed someone started a phishing attack using AdWords ads. I could keep myself awake for about 2 more hours, prevented 20k GBP of coins from being stolen, but after that I had to go to sleep with no idea of what was happening. I woke up and solved everything in 5 minutes. You really don't want a sleepy guy to perform bitcoin tasks.

      "If you'd know the nature of business I am in, you'd know that every second counts."

      Yeah, but if it counts on Localbitcoins or a random BTC site, it's not a very stable business you are in. This is an implied risk that you like to pretend isn't there, but it is. Localbitcoins just do their own thing. If you build a business where people's lives depend on 24/7 stable access to LBC, that's YOUR problem, it's NOT Localbitcoins killing people.

      "Best believe me its over with for the hacker."

      I'd pass on believing you because previously you suggested LBC should just restart the server like nothing happened and 'perform tests' while it running live, possibly with a backdoor installed. The only reason it's "over with for the hacker" is that LBC did this properly.

      Delete
    6. "If you build a business where people's lives depend on 24/7 stable access to LBC..."
      Not to LBC, just to BTC. My coins are stuck in the system. I just needed a fast exchange site. Thats what I got. :D

      "You really don't want a sleepy guy to perform bitcoin tasks."
      I dont want a sleepy guy to fix this. I want a team to fix the problem, working on it 24/7.
      As long as LBC would be a simple exchange service, what you were saying would be right, but they are an ESCROW service, so they hold our funds and they have the responsibility, thats why we use Escrow, to keep our money safe. I believe that they are doing their best to fix the issue, but at least they could update us on what is going on. Its been already 24 hours now, since their last message.

      Delete
    7. "Not to LBC, just to BTC."

      You were complaining about LBC being down, not BTC being down. So yeah, you're basing your business partly on LBC, therefore you should be aware that downtimes are a possibility. Even banks have downtime. LBC is just a bitcoin site, so your expectation should be lower.

      "I dont want a sleepy guy to fix this. I want a team to fix the problem, working on it 24/7."


      In reality, you want the LEAST amount of people with complete access to the servers (especially in the context of cryptocurrency services). It is unrealistic even for a big company to expect any random admin that's available 24/7 to be able to reinstall everything from scratch. In the case of Localbitcoins, we're talking about probably two people, and it's not out of the question that they have to work together in a case such as what just happened. Even in large data centres, above a certain level of access and for certain defined tasks, the 'staff' will be reduced to 1 or 2 people who know what to do and how to do it properly.


      "As long as LBC would be a simple exchange service, what you were saying would be right, but they are an ESCROW service"


      You've got it backwards. A proper exchange where you can deposit actual fiat money would be expected to be held to higher standards. Localbitcoins doesn't hold any fiat, they just keep the BTC you deposit. They also do escrow, but that doesn't mean you can make up your own desired level of availability and expectations and hold LBC to it.

      Regardless of whether it's an exchange, marketplace, web wallet, escrow or whatever, you don't define the terms. THE SITE DOES.

      These are the terms that once you have accepted but seem to have forgotten since:

      "Your use of this site is at your own risk. LocalBitcoins.com and all the materials, information, software, facilities, services and other content are provided 'As Is' and 'As Available' without warranties of any kind, either expresses or implied. LocalBitcoins.com does not warrant that the functions contained in this site will be available, uninterrupted or error-free, that defects will be corrected, or that LocalBitcoins.com or the servers that make them available are free of viruses or other harmful components. LocalBitcoins.com does not warrant or make any representation."

      "LocalBitcoins.Com does not accept any liability for any loss or damage, direct or indirect, resulting from any use of, or inability to use, LocalBitcoins.Com or the material, information, software, facilities, services or other content on LocalBitcoins.Com, regardless of the basis upon which liability is claimed and even if LocalBitcoins.Com has been advised of the possibility of such loss or damage. Without limitation, you (and not LocalBitcoins.Com) assume the entire cost of all necessary servicing, repair or correction or correction in the event of any such loss or damage arising."


      Do you understand now?

      Delete
  9. your all cocksucking tarts. just chill and we can all fuck the government with our non-taxable currency once again!

    ReplyDelete
  10. thanks localbitcoins team for the hard work and effort keeping the bitcoin community growing up !

    ReplyDelete
  11. I hope you guys take care of all the issues. the people at http://www.bitcoinvalues.net reported this as of it's still going on. What's the deal?

    ReplyDelete
  12. It is good to have most of these articles around to maintain the regular flow of information. Help people that no one could do it later, good work.

    Linux Thin Client & Citrix Thin Client

    ReplyDelete

Note: Only a member of this blog may post a comment.