Archive for the ‘Service’ Category
An apology, an explanation and a commitment
In the last 30 days Yola has had far too many incidents of downtime. The engineers at Yola who create the systems and take responsibility for them being up and available find this unacceptable.
Let me highlight the area where we performed worst in terms of uptime – the Published Sites. This means your site that you have created and host with us. In the past 30 days, if you have a live site, it has been unavailable for a total of 3 hours. The longest time it was unavailable was 43 minutes, and the shortest time was just one minute.
As engineers, we take the uptime of your site very seriously, even personally. We are extremely proud when we get to the end of a month and there has been no downtime. It is usually the area where we have the least downtime – because we put an extraordinary amount of effort into keeping your site up and fast. We know what it means when your site is unavailable – it could be at a time critical to your business or personal brand and you could be losing money. When your site is unavailable you feel helpless and frustrated.
As the engineer who takes responsibility for everyone involved in keeping your site up, I would like to apologize to you, explain why it has been such a rough month and tell you what we are doing to make sure it never happens again.
There has been no disaster in our service infrastructure, as you might expect after such a bad month. In fact most of the issues have been unpredicted (and honestly, completely unlikely) side effects of some real, meaningful upgrades to the infrastructure that runs our millions of hosted sites. There is always a very small chance that no matter how well you plan for upgrades to happen ‘transparently’ and professionally, something could go wrong. It feels like we have have been quite unlucky in this regard.
The upgrades we have been doing involve a process of removing our reliance on infrastructure that is known to fail sometimes, so that we are left with infrastructure that can cope when things go wrong and does not cause downtime. But more than once in the last month, that failure-prone infrastructure would fail as we were removing it, causing downtime. For those of you interested in detail, the issues have included operating system failures and network filesystem failures.
Here is a commitment to you.
- Your site’s availability and uptime is our number one priority.
- Your site data is safe and secure. We take care of backing it up and making sure we can recover it in the event of things like natural disasters.
- We will continue to improve your site’s uptime and response time.
- We strive for 100% uptime on your site.
- The same applies to our Site Builder and MyYola, although the uptime of your site will always take precedence over these if we need to make that call.
Please contact me if you have any questions whatsoever.
Lisa Retief
VP of Engineering at Yola
Labels: Operations,Service,Upgrades | 12 Comments »
User URLs Not Changing
We’ve had a lot of questions about whether the URLs of all of our users’ sites will be changing when we switch our name from SynthaSite to Yola.
Rest assured that the URLs for all of our users’ sites will stay the same. If your URL today is mysite.synthasite.com, your URL next month (after we’ve changed to Yola) will still be mysite.synthasite.com.
Hope that helps allay any concerns. We want our name change to have as little impact as possible on our user community.
Labels: Community,Releases,Service | 19 Comments »
Update on last night’s downtime
At around 21h00 PST on Wednesday night 30 July, our Site Builder
service became unavailable to users. This was due to a hard disk
failure on a critical server. There was a backup drive waiting to take
over, but for a reason still unknown to us, it did not happen. The
Site Builder (the service you log in to in order to create, edit and
publish sites) was down for about two hours while we diagnosed the
problem and switched over to our failover systems. The published sites
were never unavailable so there was no disruption for the users of
your sites.
Hardware failure is something we know will happen from time to time,
no matter how many preventative measures are put in place. So as
engineers we plan and rehearse for it so that we are able to respond
with least disruption to our user base. In this instance the longest
part of the downtime was spent making sure we knew exactly what had
gone wrong so that we could make the correct decisions for recovery
without compromising the integrity of any data.
I want to extend an apology to all users who were unable to log in
during this time. During such incidents you are foremost in our minds
while we work to bring everything online again.
SynthaSite is uncompromising in our commitment to optimal availability
and reliability of the platforms that run both the Site Builder and
the sites you host with us. We have asked the data centre where the
server is hosted to investigate the reason for it not automatically
failing over to another drive, and will take steps to prevent this
from happening in the future.
Labels: Operations,Service | 4 Comments »
Scheduled Maintenance
We’ll be upgrading our servers on Thursday, July 17th between 1am and 5am PST. During this time you will not be able to publish any changes you make to your site. You will still be able to work on your site and save your changes, but will have to come back after 5am PST to publish your site so those changes will be updated on your live website.
If you already have a published website, it will stay up during this time and your visitors will not be affected.
Thanks for your patience while we make this upgrade and thanks for using SynthaSite!
Labels: Service | 21 Comments »
Planned maintenance
SynthaSite is upgrading a server component on Friday (20 June 2008) morning and need to take the SynthaSite Site Builder offline between 1am and 2am PST. During this time, you will not be able to log in and make changes or updates to your sites. Your published sites will be unaffected.
Labels: Operations,Service,Upgrades | 6 Comments »
Maintenance completed
We have completed the scheduled maintenance and the SynthaSite services are back in working order. It took a little longer that initially anticipated – thank you for your patience.
As promised there was no downtime on the published (hosted) sites.
We are aware of a known issue which occurred as a result of the changes we made – image editing from within the Site Builder will currently not work. This is being worked on and we will let you know when it is resolved.
Labels: Operations,Service | 7 Comments »
Maintenance on Thursday morning
We’re beefing up our servers on Thursday (12 June 2008) morning and need to take SynthaSite offline between 1am and 4am PST as part of this upgrade. During this time, you will not be able to make changes or updates to your sites. But don’t worry, visitors to your published sites will not have any interruption and will still be able to see all the great stuff you’ve been publishing.
Thanks for your patience as we scale SynthaSite to meet the needs or our ever growing user community!
Labels: Operations,Service,Upgrades | 4 Comments »
Service disruption within the Site Builder
Between about 10h00 and 13h00 PST on Saturday 1 March, users trying to create, edit or publish sites would have experienced service interruptions.
This was due to a failure in DNS (Domain Name Services) that are managed at a third-party service provider.
We apologize for the disruption and want to assure you that we have taken precautions to prevent a recurrence of the problem.
Thanks to those of you who mailed to let us know that you were having trouble – we appreciate hearing from you about all aspects of the SynthaSite experience. In this case we were immediately aware of the problem and acting to fix it, but it is good to know that our users demand excellent service.
The reliability of both our Site Builder tool and hosting service is of utmost importance to us, and we have an excellent team dedicated to constantly monitoring and improving the uptime and response time of the services. We have some exciting plans underway to bring you additional features and reliability that you will be hard-pressed to find at a paid-for service, but with SynthaSite they will remain free.
Labels: Operations,Service | 13 Comments »