New VeraCrypt volume auto-dismounts seconds after file copy begins

Topics: Technical Issues
Jan 14, 2015 at 2:56 PM
I have a brand-new, 4 TB Seagate USB3 external hard drive that I recently encrypted fresh (formatted clean). I then dragged over (copy) a folder that is 1.44 TB in size.

The filecopy got as far as creating the new folder on the VeraCrypt volume, then it auto-dismounted seconds later with a message stating such from the icon in my system tray.

I'm running Win 7 ultimate, 16 GB RAM with a 6-core AMD CPU.

I tried the filecopy with the Seagate drive connected through a USB3 hub, and also connected through a USB3 port on the back of the computer. Both ended the same way, seconds later with an auto-dismount message.

Any ideas what is going on? I assume there is no log file to check.

I'm using the latest VeraCrypt installer.

Thanks in advance,
-Craig
Coordinator
Jan 14, 2015 at 8:37 PM
Hi Craig,

Can you please check Windows events log (through the EventView) to see what happened during the time the drive auto-dismounted?

Moreover, just to check, is the disk dismounted only in VeraCrypt or is it disconnected from the system as if you removed the cable? If it is dismounted only in VeraCrypt, could you please disable all auto-dismount options in VeraCrypt preferences, restart the machine and retry?

Thank you in advance for your help investigating this issue.

Jan 15, 2015 at 10:07 PM
Hi idrassi,

Thank you for your contributions to this project :-)

Here is a screenshot of the relevant entries in EventViewer for the issue I described:
Image

The warnings have this text associated:
"The system failed to flush data to the transaction log. Corruption may occur."

The Error had this text:
"The default transaction resource manager on volume M: encountered a non-retryable error and could not start. The data contains the error code."

I have not yet had a chance to try your second suggestion but will do that soon and follow up with you here.

If you need more info from the Event Viewer, please let me know.

Thanks so much,
-Craig
Jan 28, 2015 at 1:58 AM
Hello, I just want to let you know I am experiencing the same problem. 4.5tb external hhd.
I checked all dismount option and restarted and it is still occurring.
I tried doing small increments, few gb each time and the dismount persists.
I am going to try to remove the encryption to test if the hhd is the culprit.
Coordinator
Jan 28, 2015 at 3:09 AM
Thank you all for your feedback.
It seems that the error happens at the level of Windows NTFS driver who is responsible for handling the file system of the disk.

Indeed a good test would be to see if the same happens when copying data without the encryption. If without encryption everything is fine, this would mean that there may be an issue in handling such big disks.

Another test if it is possible: If you have access to a Linux machine where NTFS read/write is supported (latest distributions like Ubuntu), could you try using VeraCrypt to copy data to the disk? I have implemented a fix for Linux and MacOSX in order to support hard drives with large sector size and maybe Windows is suffering from a similar issue although it was never reported before.
Jan 28, 2015 at 3:53 AM
Edited Jan 28, 2015 at 4:13 AM
I have implemented a fix for Linux and MacOSX in order to support hard drives with large sector size and maybe Windows is suffering from a similar issue although it was never reported before.
.
Mounir, would it help you if crimius and Noplay provided their NTFS Bytes Per sizes for their external drives that are the target volumes for the data copies?

For Windows, you can get the current NTFS Bytes Per sizes using the following command:

Run as Administrator at the Command Prompt:
fsutil fsinfo ntfsinfo <Drive Letter>:

Example:

fsutil fsinfo ntfsinfo C:
Snipped output to focus on these four results:

Bytes Per Sector : 512
Bytes Per Physical Sector : 512
Bytes Per Cluster : 4096
Bytes Per FileRecord Segment : 1024
Jan 28, 2015 at 1:11 PM
I apologize for falling off this thread.

I changed my Veracrypt preferences to this:

Image

Ultimately, I was able to copy at least a terabyte of data to my external, encrypted, 4TB, usb3 drive. I also was able to access that data repeatedly without issue.

Soon after though, I had an unrelated problem with my Flexraid tRaid array and spent several days recovering/restoring a very large amount of data from backups. I re-purposed my external, encrypted, 4TB, usb3 drive for this recovery effort (without encryption) - so my experiment and testing of the Veracrypt application has been put on the backburner.

In any event, here is the command line output on my machine for the drive in question:
fsutil fsinfo ntfsinfo V:

Bytes Per Sector  :               4096
Bytes Per Physical Sector :       4096
Bytes Per Cluster :               4096
Bytes Per FileRecord Segment    : 4096
I am almost to a point where I can again use the 4TB external drive for testing Veracrypt, so please let me know if I can help with anything else.

-Craig
Jan 28, 2015 at 1:14 PM
Also, the implication that I was able to copy ~2.5 TB+ of data to my external hard drive without encryption successfully means that there may be an issue with Veracrypt handling big disks in Windows.
Coordinator
Jan 31, 2015 at 9:15 PM
Edited Jan 31, 2015 at 9:17 PM
The Windows code that handles low level read/write operations always uses 512 bytes as a unit and it is not easy to modify it to use other sector size values (4096 bytes in your case). On Linux/MacOSX, the code is simpler (no device driver equivalent) and that's why it was easy to modify it.

The fsutil output for your disk shows that it does not support the Advanced Format 512e (this is the standard emulation that enables drives to be compatible with 512 bytes based read/write operations). If this emulation was on, then the first line (Bytes Per Sector) should have shown 512 instead of 4096. More information can be found on https://en.wikipedia.org/wiki/Advanced_Format.

The modification of VeraCrypt on Windows to support sector size other than 512 will take some time. Till then, big disks with no 512-bytes emulation (e.i. 4K native) will have stability issues on Windows.
Jan 31, 2015 at 11:18 PM
Edited Jan 31, 2015 at 11:45 PM
Crimius and Noplay, you may be able to use the 4 TB drives for encryption depending on how your manufacturer preformats the drive and their default Bytes Per Sector size.

See my posting for the steps I took for a 4 TB WD external drive at this posting.

For tracking purposes, I have create an issue ticket at the following link.

https://veracrypt.codeplex.com/workitem/85
Mar 16, 2015 at 5:55 PM
Hi all!
I have an issue with both TrueCrypt and VeraCrypt which I believe must have something to do with the issue reported here.
I have a 3 TB internal drive which I have encrypted using TrueCrypt in RAW mode (i.e. the whole HDD, no partitions). The drive can be mounted just fine as long as I connect it to my desktop directly using SATA but recently I bought a Seagate Backup Plus enclosure (just the enclosure) and put the 3 TB drive in the enclosure. Now, if I connect the enclosure to my PC through USB, the drive is detected with no issues but if I try mounting it with either TrueCrypt or VeraCrypt, I get "Error: Incorrect/Invalid parameter". There are few similar reports on the internet but no one seems to have a solution, I believe it is caused by the fact that the Seagate enclosure is reporting the sector size as 4k while the sector size in the partition header is 512B. Is there any way to get around this issue?
Mar 16, 2015 at 6:30 PM
What are the results as shown in the posts above when running the fsutil commands when the drive is internal to your PC and then when the drive is in the Seagate Backup Plus attached to your PC?

Which program and version did you use to encrypt the HDD?

Did you perform an in-place encryption or format device encryption (results in loss of the existing data)?
Mar 16, 2015 at 7:06 PM
If the device is connected internally, the result is as follows:

Bytes Per Sector : 512
Bytes Per Physical Sector : <Not Supported>
Bytes Per Cluster : 4096
Bytes Per FileRecord Segment : 1024

If the device is connected externally, I can't mount it, so I can't use fsutil on it.

The disk was encrypted with TrueCrypt v7.1a and I used quick formatting when encrypting it because it was new and had no data on it.
Mar 16, 2015 at 7:35 PM
Something is not correct with the Bytes Per Physical Sector : <Not Supported> in the output.

What version of Windows?

You might want to Google search on "Bytes Per Physical Sector : <Not Supported>". Here are a couple MS links when the device reports Not Supported.

http://support.microsoft.com/kb/2510009

http://msdn.microsoft.com/en-us/library/windows/desktop/hh848035(v=vs.85).aspx
Mar 16, 2015 at 8:06 PM
Windows 7 SP1 x64, fully updated, KB982018 (Compatibility with 4k sector size) is already installed. I believe "Bytes Per Physical Sector : <Not Supported>" must have something to do with the disk having no actual partitions or a partition table windows can detect.
Coordinator
Mar 16, 2015 at 9:33 PM
The <Not Supported> string is due to VeraCrypt driver as it doesn't support the IOCTL_STORAGE_QUERY_PROPERTY request that was introduced in Vista and later. It is this IOCTL that is responsible for returning the physical sector byte size.

This doesn't explain the "Error: Incorrect/Invalid parameter". In order to have more information, can you please go to Windows EventViewer and copy any message that is logged at the same time the error happen in VeraCrypt? In previous posts, this was helpful in finding various hardware issues.
Mar 17, 2015 at 4:35 PM
Edited Mar 17, 2015 at 4:39 PM
I'm afraid no entries are added to Even Viewer when that error happens (not even an "Information" entry). This is definitely a problem in TrueCrypt/VeraCrypt because if I enter the wrong password, I get the "wrong password" error but if I enter the right password, I get the "Incorrect/Invalid parameter" error and the drive is not mounted. I put another 2 TB HDD in the enclosure which had been initially formatted and partitioned (one partition) with windows and then the whole partition was encrypted with TrueCrypt (rather than RAW encryption like the 3 TB HDD); the first time I tried to mount the partition, TrueCrypt said that the header is damaged and it has used the backup header to mount the drive but after disconnecting and reconnecting the disk, the partition was mounted without any issues. The problem seems to be limited to disks that have been encrypted without a partition/partition table.

On another note, I went ahead and bought a SATA to USB3.0 converter which in fact acts like a bridge (the HDD is detected similar to a Flash Disk in windows) and connected the aforementioned 3TB HDD to my desktop using this new converter. This time, TrueCrypt mounted the drive just fine but after some testing, I noticed that not only the transfer speed is so low for USB3.0 (less than 40 MBps) but also files are being read incorrectly from the drive (pretty much everything on the drive has some kind of a CRC/MD5 hash for error detection). Unfortunately, VeraCrypt also exhibited the same behavior. Then I connected the HDD to my laptop which is much newer than my desktop and has a proper USB3.0 controller and to all my surprise, not only the transfer speed went up to 150 MBps but also testing 500-600 GB of files showed that everything is being read from the drive correctly.

In conclusion, both TrueCrypt and VeraCrypt seem to be unable to mount a disk encrypted in RAW mode over a USB enclosure while a SATA to USB bridge seems to work fine. Also there is either something seriously wrong with my desktop's USB3.0 controller (which would explain why even the 2TB HDD failed to mount properly the first time, even though transfer speed seemed to be fine after successful mounting) or there is a bug somewhere in the software-based encrypted/decryption in TrueCrypt/VeraCrypt because the only major difference between my desktop and laptop, other than the USB3.0 controller, is the fact that my desktop processor (Core i5-750), while being more than capable of simultaneously encrypting and decrypting at least 100 MBps of data with AES (I regularly transfer data between my encrypted drives on my desktop), doesn't support the AES-NI instruction for hardware acceleration.
Mar 23, 2015 at 9:45 PM
I request clarification on the problem. Does this only occur when the entire, unpartitioned disk is encrypted, as in Kachal's post above? Because I bought a 5TB external drive last week, divided it into 1TB partitions (GPT), and have not had any problems using Veracrypt to encrypt a couple of the partitions, although I've only copied about 100GB to them so far. Should I worry about losing my data when the drive gets more full?

My drive shows 4096 bytes per sector with the fsutil command given above.

Thank you.
Coordinator
Mar 23, 2015 at 11:05 PM
There are different issues that can arise from using drives with 4096 bytes sector size depending on the manufacturer of the drive and the firmware it uses:
  • Performance problem: since VeraCrypt support only 512 bytes sector size, it relies on the drive firmware to implement a compatibility mode. For read operation, there is usually no speed issue but for write operations the speed can be slow depending on the firmware implementation.
  • Stability problem: Some drive don't implement a compatibility mode and so their firmware reject any 512-bytes based operation usually causing a dismount of the drive.
  • Alignment problem: some drive require data to be aligned on 4096-bytes boundaries. Depending on the drive firmware, a non-alignment can lead to speed issues or even data corruption issues. This problem appears is RAW mode because when disk is partitioned, Windows aligned partitions on 4096-bytes boundaries.
In your case, you will not suffer from alignment issues since you use partitions. You don't seem to have stability issue since you already copied a lot of data with no issue. For the performance issue, it is up to you to do measurements.
For me, there is no risk of corruption or data loss in your case but as a precaution, I suggest you to perform a data integrity check on the 100 GB you already copied so far (you can use for example my tool dirhash to compute the hash of directories). If the check is ok (which I think it is the case), then I don't see any risk for the data copied in the future.

Anyway, once the 4096-bytes sector size support is implement, all these issues will be solved.
Mar 23, 2015 at 11:58 PM
Edited Mar 24, 2015 at 12:03 AM
Thanks very much for your reply.

Checking maximum performance on this drive is problematic, because it is a USB3 drive, and therefore limited by the cable. However, according to the graph that Windows Explorer puts up during a copy operation, copying a large (100GB) file from an internal (SATA) drive to a Veracrypt partition on the USB3 drive averages over 120 MB/s, which is as fast as TrueCrypt, and very satisfactory for me.

And thanks again for undertaking this project. People like you are among the very small percentage of humans who actually advance civilization.
Mar 25, 2015 at 6:54 PM
Edited Oct 18, 2016 at 1:54 AM
brocks wrote:
I request clarification on the problem. Does this only occur when the entire, unpartitioned disk is encrypted, as in Kachal's post above? Because I bought a 5TB external drive last week, divided it into 1TB partitions (GPT), and have not had any problems using Veracrypt to encrypt a couple of the partitions, although I've only copied about 100GB to them so far. Should I worry about losing my data when the drive gets more full?
Based on my findings, if the disk is external and it has been partitioned and encrypted while being in its enclosure, there aren't going to be any problems. I believe it is not possible to encrypt such a drive in RAW mode, as long as the enclosure reports the physical sector size as 4k, anyway.

If the disk is internal, partitioned and encrypted and then put into an enclosure, again there aren't going to be any problems, as is with the case of my 2TB drive above. I also discovered that the reason TrueCrypt failed to mount my 2TB drive the first time was that I was mounting Patition0 (as if it has been encrypted in RAW mode) rather than Partition1.

But, if the drive is internal, encrypted in RAW mode without partitions and then put into an enclosure which forces physical sector size to 4k, both TrueCrypt and VeraCrypt are going to fail to mount the drive.