Yola News 56

Recent Yola outage: Here’s what happened

Yesterday was a very difficult day for Yola and our customers. Yola’s servers were attacked unexpectedly and maliciously by as yet unknown individuals and/or groups, bringing down all of our hosted websites as well as our service provider’s entire datacenter.

The thing Yola cares about most is providing a world class service to all our customers, and doing so consistently and reliably. When our servers are attacked, your website is attacked. We don’t take this lightly. We worked around the clock to bring the service back up, and talked to you in our customer community forum, Twitter, Facebook and email during the outage to answer your questions and keep you up-to-date.

We’re posting this information on our blog now to tell you what happened, what we’re going to do about it in the future, and, most importantly, to apologize. Regardless of the reasons, your website was down. Please know that however frustrated we were with the situation, we understand you were more so. We’re very sorry.

Here’s what happened and what we plan to do about it:

What was attacked?
The IP address of our hosting service came under a vicious Denial of Service (“DoS”) attack, affecting all of our hosted sites. The attack consisted of an enormous amount of nonsense data aimed at our servers. With all of this junk flowing through it, it was impossible for legitimate traffic to get to your sites.

What happened to my website? Is it secure now?
Your website is safe. It was secure during the attack, but impossible to get to (kind of like a roadblock). We’ve created another route to your site, and have been updating all the domains we control to use this route.

What do I need to do?
If you bought your domain from Yola or you use our yolasite.com subdomain, your site is already fully functional and you don’t need to do a thing. We’ve already done everything to move your site to the new address, and everything should be up and running normally.

If you bought your domain elsewhere (i.e. through a different domain registrar) and are pointing it to our service, please see these instructions for reconfiguring your DNS: Pointing your domain to Yola.

Why did this happen to Yola?
This unfortunately happens to many companies on the web. There are just people who like to do malicious things. We’re not the first site to experience this, and we won’t be the last. We’ll continue to do everything possible to protect our service and our customers from these kinds of attacks in the future.

What did Yola have in place to protect its customers from this attack?
We’ve always had state-of-the art defenses in place; we split our networks up so that an attack on one part would not affect the other, and we bought special hardware to put in front of our sites. Normally these defenses were more than we needed. Not this time. We couldn’t have seen an attack like this coming, and what we had in place just wasn’t enough. We’re fixing this.

What will Yola do to prevent another attack of this nature in future?
We’ve brought in a global web security firm whose expertise is the prevention and mitigation of incidents like this. We’re now creating the kind of protection usually reserved only for banks and government institutions to ensure this does not happen again, and will surpass that of most website hosting services.

Who can I contact if I have questions or concerns?
You can email us at support@yola.com, or post to our forum. Our staff will get back to you as fast as they possibly can.

Thank you for your patience and support during this trying time. The questions and comments we received (even the critical ones!) confirmed what we already knew: Yola has the best customers around.

On behalf of the entire Yola team,

Vinny Lingham
CEO, Yola.com

  • nicola

    Yola team, well done for handling this really well and finding a solution in such a short time. Let’s hope this kind of thing doesn’t happen to us all again. 🙂

  • I agree with Nicola, you did such an excellent job getting our sites back up, I am extremely grateful! Unfortunately, some people on the forum just didn’t understand your efforts, and posted horrible posts which seemed to imply that it was Yola’s fault. I however, am deeply satisfied with everything Yola has done. I take my hat off to you.
    Many many thanks,
    Callum Booth

  • My account was purchased through a different company with a pointer to yola. This required a bit more work to correct. I was pleasantly surprised by the quick response of the yola support “guys” (it actually was a lady helping me) to help me specifically with that issue.
    Thanks Yola Support!

  • JG

    Thanks for the info! Love the transparency.

    One suggestion: for those of us (me) who didn’t purchase a domain through Yola and had to do a manual workaround, an email with a heads up on how to fix this would have been hugely helpful. I learned from a customer that my website was down, and only after I found the forum post about changing to the new IP was I able to restore service. I commend you on using the forums, twitter, and facebook, but the those are still relatively niche services when compared to the broad reach (and frequent checking) of email.

    Keep up the good work.

    • LauraT

      JG, we completely agree with you and are setting up an infrastructure to speed up an email announcement should anything like this ever happen again. You can add your experience to this thread on our forum.

      Our first goal is to prevent this from happening again. Our second goal is to be better prepared for the fallout if it does. We appreciate your patience and support in the midst of this terrible situation.

  • Well done guys. Had a nice day off yesterday!

    Great support in what must have been a manic time.

  • My pointer picked up the DNS update so I didn’t have to do a thing. Good Job of getting it back up in a timely manner. I hope they find the dirtbag(s) that did it.

  • Nice to see that you’s are bringing in experts to help bring your defenses up to the highest possible point. Nothing on the web is safe and 100% secure. All you can do is just try to have the best protection to help avoid these things.

    Hopefully this firm will be able to help you develop security protocols to help avoid this from happening.

    I am glad to see that the Yola staff remained claim throughout the entire thing. ESP when you get a bunch of clients that don’t understand its not something you can just stop by clicking a couple of buttons.

  • my site is still down n my customer who is interested to get some bags cant access!!!!! Please HELP.

    • LauraT

      Hi Monica, it’s 184.72.229.176. If you need instructions on pointing your domain you can find them here. Please let us know if you need any further help.

  • Btw, i purchase my domain fm another company. Can someone fm Yola pls provide me a new IP address? Tks!

  • Tks Laura! Will forward it to the domain company.
    Hope evertything goes smoothly.

  • I am still able to see my site. If you try to view it other than in the ‘yola’ window it will not load. Help.

    • LauraT

      @Flo B. I was able to view aceyoungaceaholics.com. Please post the URL if you are referring to a different site and I’ll look into it for you.

  • Hi there Yola team, EXTREMELY happy of the results and the fact that our sites will be pretected in the future like big bank institution.

    I’ll migrate other websites to Yola and build them with you guys. I think everybody should do the same! We experienced a great customer support and will be more protected in the future!

  • wow

    wow hats off to the hackers nice work

  • I have only a great deal of respect for Yola and thank you for the impeccable service you provide. I look forward t a long and properous relationship here.

  • What a pleasure it is to have a truly professional team supporting me and all the other Yola customers. Thanks guys and gals.

  • Codebreakersclub

    Have You Found The Hacker Now?

  • I’m glad that Yola were able to deal with this quickly. I’m currently using the service to aid my work into getting published – so whilst my website was down (not that I knew!) any prospective agents or publishers who were most likely NOT looking at my wesbites would have been unable to look.

    Hope everything goes smoothly, as Monica said.
    If you need help with anything, just ask (:

  • GREAT JOB. I noticed some new widgets added to the toolbar, you handled the attacking of the servers and added some new tools. Once again, GREAT JOB. By the way, I have noticed many people wanting to make mobile sites with their cellphones and their “PSP’s”. I think it imperative to add a feature to all accounts including a way to make subdomains with the above consoles. thank you VERY much.

    • LauraT

      @Aareon: Thanks! The widgets were added a week prior, we’re good but we’re not that good! :0) Appreciate your advice on a mobile app, it’s definitely on our radar.

  • tan

    the ip address has been updated however still not able to access website. please help. thanks.

  • The recent outage problems of yola,firstly I would like to thank the organisation for their time and effort with dealing with this threat, but please could they now look at individual websites to help the less technical of us and to get our websites back online, so we all can get back to normal. I understand the difficulties but how long will this outage last.

    • Monique

      @paradise: I can see your site: http://paradisefound.yolasite.com/. The outage was resolved on Wednesday 18 August. Please note that there is no ongoing outage at this time. Anyone using a yolasite subdomain or a domain purchased through us was taken care of by our engineers on the 18th. Action is only required on the part of users who bought their domains from a registrar other than Yola. This does not seem to be the case in regard to your website. If you still require individual assistance, you are welcome to email support@yola.com and we will look into it as soon as possible.

  • Thank you for your quick reply and I have been in contact with support but as yet still unable to access my site,I have done everything asked by the support team and more, but still unable to get to my website.

    You say the outage was resolved on Wednesday the 18th August, but how can this been if people are still unable to access their websites as we were before the outage.

  • Thank you Yola for your quick response. I was working on my website when it happen. I added new personalized emails addresses at the time. After doing so, Yola and my website was unaccessable. I thought I made an error entering the data. When I realize I could access other sites, I knew I was a server issue. This was about 3am or so. By noon eastern time, My website was up and going again. After reading the email you sent me in reply. I was shock to hear what happen. I understand very well the night you guys had and I praise you all for your quick response to correct this massive problem. Again, thank you…your PR response was impeccable. Keep up the good work.

    • Monique

      @Claude: Sorry for the bad experience you had after adding your new email addresses. I am glad your site is up and running again. Thank you for your support and understanding. We really appreciate it!

  • Monique

    @paradise. Your site is definitely live on the internet. Drop me an email at support@yola.com and mark it “Attention Monique” and I will send you a screenshot of your site. The only thing I can think of is that there is some very persistent caching at an ISP level which is causing you to go to the old IP address when you try to access your site. If this is the case there is very little you can do besides wait. However, please drop me an email and we can work through this problem to make sure that everything possible has been done.

  • Thank you Briannia, for your response in this matter, please could you enlighten me on the time, The cache takes to empty/or clear, as for your question over using a different computer, I think this is irrelevant as I only have access to one computer and would still have a problem with this computer, wouldn’t I ??

  • Thanks Yola for a fast response and exceptional support. Truly professional and much appreciated.

  • The Outage, Again I ask for your help in my situation as I am still suffering from The Outage from the 18th of AUGUST.I have basically been told by admin that all I should do is sit and wait for my cache to clear which takes 24 hrs or longer, but how long is longer, will I still have this problem in September ?
    They tell me my website is up and running and the problem is with my computer and I should use another computer,but I ask is this a REAL solution to the problem which is not of my making.
    I apologise for this rant, but have to get this of my chest.
    Hoping this comment moves me forward in the quest for my website back.

    • LauraT

      @ paradise: If you are referring to this site: http://paradisefound.yolasite.com/ it is up and running as has been confirmed. Please let us know if you are referring to a different site.

  • Thank you for your reply,Yes I am referring to the above site in question and I have already been in this conversation by private email, however I am still unable to access my site since ‘The Outage! of August the 18th.
    I would like to be able to return to the situation before ‘The Outage’ and return to normality,can you help me alleviate this problem.

  • Monique

    @paradise – we don’t know what else to tell you. We have been able to view your live site since 18 August and we don’t know why you are unable to do so. Can you take a screenshot of what you see when you try to open http://paradisefound.yolasite.com? Email this to support@yola.com and we will try to figure out what the problem is.

  • Jason

    I’ve been looking to try out Yola’s services- I had my website on Wix.com for almost a year.

    I was furious when this type of thing happened at Wix, but now I see that it happens everywhere you go.

    I didn’t need to wait 24 hours at Wix though, they got to it and fixed it in an hour…

  • My site is down again today. It says “Safari can’t open the page “http://www.diorcomakeup.com/” because the server where this page is located isn’t responding.” I hope this gets fixed soon and for good. It’s difficult to promote my website if it’s always down.

    • Hello there Dior, Your site seems to be working correctly from our side. Could you please email us at support@yola.com so we can troubleshoot your problem further there.

  • I sent an email to customer support again today, with my screen shot attached to let you guys see what’s showing up when I go to my site. It got back working today but tonight it’s down again! I hope ou can help me with this one as I cannot do business online with my website going on and off!

  • Great job on communicating this attack to customers!

  • imu

    I singed up only today, and I didn’t know about this until I read about it.
    I really hope this kinda thing doesn’t happen again.

    • LauraT

      Hi imu,

      We’re so glad you joined us! We wish we could guarantee that it won’t. While we can’t, what we can promise you is that we’re doing everything in our power to prevent it and to handle it better in the unfortunate instance that something like this does happen again.

  • Pingback: Yola’s Downtime Yesterday | SiteBuilder Reviews()

  • I seem to have lost my Yola site as well.

    Is there a chance I can have it back before Saturday, as it needs updating?

    Thank You.

    • LauraT

      Hi Adrian, If the instructions in this post don’t help you to republish your site, please email your domain name to support@yola.com so we can check your domain settings for you.

  • Keith Fuller

    What equipment did Yola purchase to prevent future DoS? If it was a WAF what brand was it?

    • LauraT

      Hi Keith,

      We are not at liberty to discuss the precautions we have taken other than to say that we did make some major changes and have been pleased with the results thus far. Sorry we can’t be of more help with your inquiry.

  • The site comes up, but I can make no changes or additions or retrieve my mail. What do I do? Wix was the easiest site I ever used. Can I get it back?

    Harry Emberson

    • LauraT

      Hi Harry,

      I removed your phone number from your post for your security. Can you please email support@yola.com with your site address so we have the necessary information in order to address your problem? I’m confident we can get you sorted!

  • why is the site builder down?

  • The status message:

    We’re working quickly to recover from an issue in one of our database clusters. We’re incredibly sorry for the inconvenience.

  • is this attack already over yet ? because today 22 desember 2010, i can’t get in to my site-builder for editing.

    • LauraT

      Hi rara,

      There is no attack, nor are we experiencing any server issues. If you are still struggling to access your Sitebuilder please email support@yola.com for assistance.

  • It is 12/27/10 and my website won’t work! I have to have it working by Friday! Please help!

    • LauraT

      Hi Tally, what is your site address? I suggest you email support@yola.com with your account information so we can assist you further.