| Author | Message | ||
     Udo Carl Username: blowtorch Registered: N/A |
Hallo, when i create a new file (File/New/4GB) it will be instantly created in the temp folder with that size. If i save the file it needs a very long time to allocate that 4GB again. Is it possible to leave (or move) that empty file so it remains on disk without using the current save method? It would be a big speed improvement. | ||
| Stefan Fleischmann Username: admin Registered: 1-2001 |
If you want to create a big file on NTFS instantly without initializing it, then create a small file first (File | New), save it, and then increase the file size in File | Properties (enter a larger file size and hit the enter key while the cursor is in the edit box for the file size). | ||
| Udo Carl Username: blowtorch Registered: N/A |
Thanks, working great. I guess its a security thing that there's no real 'ok' button? I've created a 100 bytes files and extended it to 4GB (worked instantly indeed). When i reopen the file everything is 0x00. How is that possible? I mean, isn't it just a pointer telling start and end position (that's why its so fast)? Shouldn't it show the actual hdd content that relies in that area? Or is it just my hdd that is really 0x00? Very unlikely. o_O | ||
| Stefan Fleischmann Username: admin Registered: 1-2001 |
The NTFS file system remembers how many bytes have actually been written to the file (100), and if the file system driver is instructed to read beyond that point, it simply delivers zero-value bytes and no longer reads from the disk. If you open such a file internally in WinHex or X-Ways Forensics (from within the directory browser), then you will see the data that is actually stored in the clusters that are now allocated to that file (of course data from previously existing files). | ||
| Udo Carl Username: blowtorch Registered: N/A |
Ah, ok. Its a NTFS sparse file then i guess. Is it also possible to create files with winhex without sparse flag (or partially initialization)? Allocate all clusters blindly so you would actually see old data in the new file (and benefit from instant file creation)? | ||
| Pánczél, Levente Username: panczel_levente Registered: N/A |
Someone please correct me if I'm wrong but in my point of view the following factors are relevant: There are two different ways how WinHex can access a disk. First, it can access the raw (sector-level) data, interpret it, and make modifications on that level. Only this method can be used to access previously existing files' contents. It's no good idea to manipulate the file system with this method (i.e. manual editing) unless one has a really good knowledge of the filesystem's internal data structures. Second, WinHex uses the same FileSystem API's that any other program uses to manipulate files (CreateFile, ReadHandle, CloseHandle, etc.). This is used in all other cases except when opening a physical disk. This method has all restrictions (and performance benefits) that Windows provides. Now let's look at the case of simple files on NTFS (i.e. no compression and no sparseness): NTFS keeps track of many metadata of files. One is logical size: this is the last byte's position (applications declare this by truncating, writing and explicitly setting). Another is initialized size: this is the position of the first byte after which no data has been written to the file yet. These two usually but not necessarily match. Logical size determines allocation (this is the point of explicitly setting file size): a file's contents may be freely modified without needing more quota unless the logical size is raised. Initialized size determines the sectors being written: only those sectors of file data have to be written to disk that are affected by initialized size. The uninitialized portion of a file (i.e. the gap between these two points) is observable (at least if it's big enough). Sector-level mode of WinHex can be used to analyze data in this section of a file. Using the FileSystem APIs to read this portion of a file is also legal; but secure operating systems (see concepts of isolating processes, users) MUST NOT allow the programs to see any uninitialized data, thus must return data that does not carry any information. This is just as true for the uninitialized portion of a file as for newly allocated memory. Since file data should be consistent with itself Windows chose to define uninitialized data as a field of 0 bytes. So reading from the uninitialized portion of a file with the FileSystem APIs always yields all-0s, this restriction comes with the use of the OS's file management. Sparseness is a technology of NTFS that is very rarely used. It says that when a range of 0s is considered "meaningless" data then it can be written to the file with a special operation and may not be allocated. Such sparse regions are always returned as a field of all-0s (and possibly without disk I/O). Note that no 0-fields are converted to sparse ranges unless the application writing them uses the designated special operation. So I think that's all needed to clarify the questions. (I will not consider the option of manually editing the filesystem structures.) "Is it also possible to create files with winhex without sparse flag (or partially initialization)?" Sparse files can only be created with WinHex if one chooses to create a raw disk image with sparse compression (I think this does not apply here). To eliminate partial initialization make sure the last logical byte is written to the file: e.g. move to the last byte of the file and update it to the same value (by changing it and then changing it back like 00->01->00) and save! "Allocate all clusters blindly so you would actually see old data in the new file (and benefit from instant file creation)?" The old data might only be seen with the first method but that has nothing to do with file contents (especially it has no connection to what file contents other application observe). If the goal is to quickly create large file that works as "a window to old file contents" for common application then I would say this is not viable through Windows' NTFS driver. But what I wrote for is that I'm curious whether others have objections or better alternatives for the next solution of this above problem. One might create a large uninitialized file [without the ability to control where it will be actually allocated] and then use sector-level access in WinHex to find the metadata of that file, and set it's initialized size to the same value as the filesize. For the latter operation the volume should be unmounted to avoid interference with the NTFS driver's internal caches. Note: Since this approach involves manual editing of a disk one should only use it with bearing the full responsibility of possibly losing all data; so I do not recommend anyone to perform it. So the safe solution is to use WinHex to open the disk, select the desired "old data" and use ( Edit | Copy Block | Into new file ) to save it to ANOTER volume. | ||
| Stefan Fleischmann Username: admin Registered: 1-2001 |
> Its a NTFS sparse file then i guess. No, it is not, that is yet another concept. | ||
| Udo Carl Username: blowtorch Registered: N/A |
Thank you for the explanation. |