I believe that this has always been the problem, trim simply does not work with raid 1. I intend to rerun the test but I'm quite sure the results will be the same.

Just to update, A push of the reset button while everything was running also completed successfully. After restarting, parity verification showed no errors. I am now about to run another 5 day test having swapped out the intel 530 240gb for intel s3710 200gbs. I will report the results back next weekend.

Just reported all results to Intel engineers so that maybe they can help.

Very interesting!
Just to discuss the basics: Does anybody have a whole understanding about SSDs TRIM?
If my understanding is correct, the RAID Driver has (simply) to 'duplicate' the TRIM command in RAID 1 and has to send it to both SSDs (that's what I never understood, why TRIM was not supported by the drivers for such a long time).
One of my theories ever was that the TRIM command has different Impacts on both SSDs, because it's simply different hardware...
Maybe the verification faults resulting of the different content when reading the raw sectors of the SSDs, which are TRIMed (means Contents does not care for the file System)?
Because I never had serious file system corruption!
Regards

Hi Billydv,

I think you have just performed the most interesting test so far - thank you for that!
I hope you have more response from Intel than we did. We only managed some vague Chinese whispers via SuperMicro. None of which fixed anything.
But I will keep my finger crossed... for all of us!
Good luck!

Steve

Hi Stephen,
I could have stopped simply at the point of knowing that 3.6 works with server 2008r2 but I have a big problem, I will be going to server 2012r2 which according to intel requires at a min 3.8 drivers. Now that I have repeatable testing that shows the problem is with the enabling of trim, I know that I can install 3.8 drivers on 2012r2 and simply disable trim.

Just wanted to show everyone some emails exchanged between Intel and myself regarding sata OROM version and IRSTe driver versions. Stephen, this is your exact situation.
My email to Intel:
If I could just get one definitive answer in the interim, Since the 3.6 series IRSTe are the drivers recommended for server 2008r2 on your website, can they be used successfully on a mboard that has sata orom 3.8.0.1029? What if any disadvantage/danger will there be if I was to use that configuration?
Intel's response:
Hello Bill,
There should not be any problem if you use an Intel RSTe version older
than the Option ROM version. Intel RST should be able to work fine with
newer and older OROM versions because it has been validated for the SATA
controller that comes in that motherboard.

@billydv
Thank you for this detailed info!
Yes, you are right, I did not realize, that the 13.6 Drivers are not made to Support my C204 chipset (I simply updated to the newest Version I found @Intel, because I had some other Problems which I hoped to solve with this update - and because it did instal: ...)
Actually I finished my stress tests using the 12.8.0.1016 driver (supermicro recommends) and what should I say: it works without any verification faults!

Now I do my next steps, because I have to migrate some Servers from 2008r2 to 2012r2. The problem is, that the 12.8.0.1016 Driver seems not to support 2012r2 (following its readme). I installed it anyway on a test machine running 2012r2 and my first impression is that anything is working fine.

Any comments on this?

Thank you an advance!

Just a copy of my email to SuperMicro support Europe with an already open ticket regarding this matter.

Hi Quintijn,
I just wanted to give you and your company some info as I have been testing extensively to find the exact cause to our problems. Ultimately what I have found is that while using ssds in a raid 1 configuration, only the 3.6 series drivers from Intel's website work correctly with no parity errors or mirror corruption under high IO load. Stress testing is done via Passmark Burn in test with disk testing set to 60%. Also, a push of the reset button while system is running also does not damage the volume whatsoever, parity is maintained and verification shows no errors. The main difference with 3.6 series IRSTe drivers is that they do not enable trim in raid 1.
I have also tested extensively with the 3.8 series drivers from Intel's website. If the driver is installed and used normally, ssd raid 1 volumes will corrupt. If trim is disabled in OS via "fsutil behavior set disabledeletenotify 1", the 3.8 series drivers are stable, again with both manners of tests, Passmark and a push of the reset button. Neither test showed any errors in parity verification. What I believe is most important for you to know is that my testing with the 4.2 series drivers fails even with hdds. Currently you have 4.2 series drivers on your website as recommended.

Here are detailed results
Server 2008r2, C602 Chipset

1- IRSTe 3.8, 3.9, 4.1, 4.2 with raid 1 ssds fails

2- IRSTe 3.6, 3.8, 3.9, 4.1 with raid 1 hdds succeeds

3- IRSTe 4.2 with raid 1 hdds fails

4- IRSTe 3.6 with raid 1 ssds succeeds (I ran this twice to confirm my results, each time resulted in no parity errors. A 3rd test was to push the reset button on the server while it was running, Server restarted normal and parity verification reported no errors)

5- IRSTe 3.8 with raid 1 ssds succeeds if trim is disabled via "fsutil behavior set disabledeletenotify 1"

The above tests are repeatable. I believe ultimately that trim in raid 1 with ssds is simply broken in spite of the fact that all IRSTe drivers version 3.8 and higher support and enable it. I believe the tests would be repeatable in all motherboards with c600 series chipsets. I would hope that your engineers could discuss the matter with Intel engineers to correct the issue and ultimately either fix trim in raid 1 or simply update all driver versions to disable it. Please keep me posted of any changes regarding this situation.
Thanks,
Billy DeVincentis

The RealBibo,
If you now are tesing Server 2012r2, have any of Intels 12 series IRST drivers installed? I would probably test extensively with the 12.9 series
https://downloadcenter.intel.com/downloa...RST-RAID-Driver
I believe on your chipset trim in raid 1 is disabled anyway so hopefully you would not see any of these issues. Test both the 12.9 and 13.6 series. If you see any parity issues, disable trim in OS and test again. As I told you previously, I have countless workstations running ssds in raid 1 with 13.6 series drivers and have never had any issues, trim is disabled by default on those systems.

The Real Bibo
See this here https://communities.intel.com/message/273293
Someone else is confirming that trim is not enabled in raid 1 with IRST

Just wanted to mention that I was looking at Intel's download center for IRSTe and they now have the 3.5 series driver as recommended for server 2008r2. The 3.6 series is only recommended for web server 2008r2. Not sure if it's a mixup or not but I haven't seen a single reason not to be using the 3.6 series on server 2008r2. I now have that version installed on our primary server with hdds and also on our backup server with ssds. I currently have the backup server running a 30 day stress test similar to what I had run in my previous 4-5 day testing with the various driver versions. As soon as I see that even a 30 day stress test will not corrupt, I will switch to ssds on our primary server.
BTW, here is the email I received from SuperMicro, they finally are admitting that others are having similar issues with ssds in raid 1
Hi Billy,
Thanks for your feedback.
Incidently we also received similar feedback from another customer today informing that somehow TRIM seems to cause data corruption in the RAID set with SSD’s.
We will forward this information to the PM for further investigation, if Supermicro can do something to solve this we will of course come with a new BIOS.

@billydv

Zitat von billydv im Beitrag #100

---

The RealBibo,
If you now are tesing Server 2012r2, have any of Intels 12 series IRST drivers installed? I would probably test extensively with the 12.9 series
https://downloadcenter.intel.com/downloa...RST-RAID-Driver
I believe on your chipset trim in raid 1 is disabled anyway so hopefully you would not see any of these issues. Test both the 12.9 and 13.6 series. If you see any parity issues, disable trim in OS and test again. As I told you previously, I have countless workstations running ssds in raid 1 with 13.6 series drivers and have never had any issues, trim is disabled by default on those systems.

---

We know that trim in raid 1 just doesn't work. I would use the recommended driver for your OS with your sata chipset and just disable trim straight from the OS.
fsutil behavior set disabledeletenotify 1

Okay,
Now I have begun testing the very same server with new OS, Win Server 2012r2. I am using the 3.8 Intel drivers direct from Intel's website. Trim is enabled and working as per trimcheck 0.7, I have begun the same stress test with Passmark Burn In test. I will report back next weekend as the test will go for 5 days.