VMware vSAN: Troubleshooting Workshop [6.x] Beta experience Day 1

Today I attended the first out of two days of my first VMware Beta course. This time it was time to add vSAN to my portfolio. I think first of all a small disclaimer is in order. As said the workshop is in beta. This article is based upon my personal experience. Content might change later on as might the number of days.

The course  expects that you have basic knowledge and understanding of the vSAN product. The following course is highly recommended before you attend this workshop.

VMware vSAN: Deploy and Manage [V6.6]

The  workshop is a two days course and is delivered via Instructor Led Training (Live Online). It consists of a virtual lab which is located in the United States. People who have followed courses from VMware before might be familiar with this concept since VMware offers these labs during more of their courses. This with exempt to training centers that host and build their own labs.

The course consists of four modules:

Module 1: Course introduction

Module 2: vSAN Software Architecture

Module 3: Troubleshooting Methodology

Module 4: Troubleshooting Tools

One of the best parts of the course is that it includes 14 labs of which 10 Break Fix labs (Not including the 11th bonus Break Fix). Based on real life experience, most common errors are being reproduced and replicated in your lab environment which you have to fix in a calm and controlled environment.

The course is given by Javier Menendez who also happens to have a blog which has some nice detailed information on how the vSAN commands cmmds-tool and objtool work.  Of course these commands come with the necessary warnings about incorrect use and potential damage.

The day started with the vSAN architecture explaining CLOM, DOM, LSOM and CMMDS. Also FTT (Failures To Tolerate), which is now called PFTT (Primary Failures To Tolerate) was discussed and how many objects are being created on the vSAN datastore.

To conclude the day a beginning was made with Module 3: Troubleshooting Methodology.

Please visit back later to learn more about day two.



Synology DSM update to 6.1.3 update 6 HTTP Error 500

Like many others of you I own a Synology which I use to run a various load of servers. It is my trustworthy friend which is delivering NAS based storage to my VMware environment for quite some years now. Over the years I have been exploring the Synology quite a bit more and learned to appreciate many other features as well like “Web Station” which I use to host some WordPress websites.

Synology Diskstation DS2514+

There is one thing that I dread however. Quite ever so often Synology releases a new version of the Disk Station Manager (DSM). New featrures are added, old features are discontinued and security fixes are applied. This is also the case for the numerous packages that are available in Package Center. All nice and perfect you would say. Great job Synology! However this time I was presented with the following screen when trying to reach the websites I host on the box. All websites seemed to have been broken somehow and present the “HTTP ERROR 500” message.

Of course these things happen at the most inconvenient of times. Since I did not want to present visitors to the webpage with an HTTP ERROR 500 I decided to give Google a try and search for the error. Unfortunately it seems to be the most common error you can find so the resolution is not dictated and spelled out for you. However some peoples PHP Extensions seems to have been messed up by the DSM upgrade so I decided to give look at my PHP settings first.


Click to enlarge

In my case where I use PHP 5.6 it seems the update messed with my PHP extensions. As you can see the “mysqli” extension checkbox is unmarked. Since WordPress makes use of the “MariaDB” database which is a fork of MySQL all started to make sense now.

Click to enlarge


Just re-enable the extension, click “OK”and presto! You should be able to give your website a go again!

Synology owners beware: Synology DSM Update 6.1.3 to Fortify Data Integrity

As received today from the Synology Development team:

Dear Synology users,

An important update for the DiskStation Manager (DSM) 6.1 is released. We suggest users to update your DSM as soon as possible to ensure the integrity of your data. Download the update on your DSM: Control Panel > Update & Restore.

An open source issue that’s prone to affect data integrity in rare situations has been identified. Although it only affects a handful of users, users are encouraged to upgrade to the latest DSM 6.1.3 update 4 and conduct RAID scrubbing to avoid running into this issue.

The issue may occur only when the following conditions are all met:

  • The server is using RAID 5, RAID 6, RAID F1, or SHR with more than 3 disks.
  • The volume is created on EXT4 file system.
  • The server is running on DSM 6.0.2-8451 and above.

If your DiskStation meets the criteria, please conduct the following steps:

  1. Upgrade to DSM 6.1.3 update 4 (Download now)
  2. Conduct RAID scrubbing. (Learn more)

The issue was brought to our attention a few weeks ago when the NAS communities discussed the potential data integrity threat that may occur after a RAID 5 repair. In-house investigation revealed the issue stems from Linux feature known as skip_copy. This open source feature contains a slight design fault that may lead to potential data corruption after a RAID array is repaired from degrade mode. Synology team moved quickly to identify and reproduce the issue, which is resolved in the latest DSM 6.1.3 update.

Synology is committed to high standards of data protection and safety. Keeping user information safe has always been our priority. In addition, as a vibrant supporter of Open Source, Synology has reported this issue to the Community. By calling to the attention of the mass community, we seek to join forces with influencers in the field in delivering continuous contribution on data integrity and data protection.

Owners with the following models should upgrade their models to the DSM 6.1.3 update 4 as soon as possible:

17 series: RS18017xs+, RS4017xs+, RS3617xs+, RS3617xs, RS3617RPxs, DS3617xs, DS1817+, DS1817, DS1517+, DS1517
16 series: RS18016xs+, RS2416+, RS2416RP+, RS816, DS916+, DS716+II, DS716+, DS416play, DS416slim, DS416j, DS416
15 series: RS815+, RS815RP+, RS815, RC18015xs+, DS3615xs, DS2415+, DS2015xs, DS1815+, DS1515+, DS1515, DS715, DS415+, DS415play
14 series: RS3614xs+, RS3614xs, RS3614RPxs, RS2414+, RS2414RP+, RS814+, RS814RP+, RS814, RS214, DS414slim, DS414j, DS414
13 series: RS10613xs+, RS3413xs+, DS2413+, DS1813+, DS1513+, DS713+, DS413j, DS413
12 series: RS3412xs, RS3412RPxs, RS2212+, RS2212RP+, RS812+, RS812RP+, RS812, DS3612xs, DS1812+, DS1512+, DS712+, DS412+
11 series: RS3411xs, RS3411RPxs, RS2211+, RS2211RP+, RS411, DS3611xs, DS2411+, DS1511+, DS411+II, DS411+, DS411slim, DS411j, DS411
Synology Development Team