tag:blogger.com,1999:blog-8511499809446054798.post3359886939711878596..comments2023-03-01T04:43:24.404-08:00Comments on Linux HowTo: Increase USB Flash Drive Write SpeedMichaelhttp://www.blogger.com/profile/09683047404854369708noreply@blogger.comBlogger42125tag:blogger.com,1999:blog-8511499809446054798.post-54469918626097636432023-03-01T04:43:24.404-08:002023-03-01T04:43:24.404-08:00Hello, can you increase the speed formatted in fat...Hello, can you increase the speed formatted in fat32?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-48058740482611941712022-10-31T03:40:19.913-07:002022-10-31T03:40:19.913-07:00I read your blog now share great information here....I read your blog now share great information here. <a href="https://www.archiveon.cloud/" rel="nofollow"><b>Salesforce data backup and recovery</b></a><br />archiveoncloudhttps://www.blogger.com/profile/12057309701470532651noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-68444063258175167852019-05-05T13:28:29.237-07:002019-05-05T13:28:29.237-07:00I am very enjoyed for this blog. Its an informativ...I am very enjoyed for this blog. Its an informative topic. It help me very much to solve some problems. Its opportunity are so fantastic and working style so speedy. <a href="https://engravedheaven.co.uk/" rel="nofollow">wooden wedding usb</a><br />James leohttps://www.blogger.com/profile/13251957090067175456noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-64830280222778068682017-03-03T12:09:18.389-08:002017-03-03T12:09:18.389-08:00This flash is seen under android tablet, and ancie...This flash is seen under android tablet, and ancient laptop with winxp.<br />Copying in xp from flash to flash has 1.8mb/s.<br /><br />Addition: write speed i measured by simply copying 9gb of large movie files from external usb hdd to flash.<br /><br />I also reserved 512 clusters after fat for vfat data alignment.<br /><br /><br />I made partitioning with gparted and dont remember drive geometry or whether lba mode is used.<br /><br />Can modifying drive geometry/setting lba mode help to speed up this flash?<br /><br />Why is this flash is not seen by Vista?<br /><br />What can I try more in order to speed it up to normal 7mb/s, shown by flashbench?<br /><br />Thanks to all in advance,<br />IgorAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-12630286251193619882017-03-03T11:56:04.306-08:002017-03-03T11:56:04.306-08:00Well,
while you are wondering here how factory-mad...Well,<br />while you are wondering here how factory-made partition and fs layout may not be optimal I faced such a problem.<br />My wife bought Silicon power luxmini 32gb fladh drive with significant discount.<br /><br />And write speed is nightmare 2mbyte/s.<br />Its default layout:the only fat32 partition starts at 1.28mb!!!! Not 1,2,4 mb, not multiple of 64k or so!!!!!<br />512b sector size<br />4k cluster<br /><br />After reading lot of howtos <br />I<br />tested it with flashbench and found peaks of performance at <br />64k and 4m block sizes at 7 mbytes/s. First should be pagesize×"number of parallel pages could be read/written"<br />Second probably allocation group.<br /><br />Alignment of partition start to 4m<br />+ creating fat32 with 64k cluster size and alignment of its start of data section to 4m made large progress to copying files.<br /><br />It is 3.5mbytes/s now under linux but this flash is not seen in win Vista:angry smile:.<br /><br /><br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-63309507066363593592017-03-01T06:41:54.150-08:002017-03-01T06:41:54.150-08:00Since 2002, the CHS geometry addressing method is ...Since 2002, the CHS geometry addressing method is obsolete. I recommend to refrain from using it. Besides, it is intuitive only with regard to HDD; it does not make sense to use it with flash memories. Use LBA (logical block addressing) exclusively, and work with linear sectors and clusters.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-89874584023250574372016-02-26T01:28:43.675-08:002016-02-26T01:28:43.675-08:00That's interesting. But I've read so many ...That's interesting. But I've read so many times that the pre-installed filesystem on a usb-stick usually have the best optimized parameters. Isn't that the case?Hi-Angelhttps://www.blogger.com/profile/10661636579329649996noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-13120518203802501692014-10-21T06:31:15.387-07:002014-10-21T06:31:15.387-07:00thx for the tutorial.im a new user too on ubuntu.b...thx for the tutorial.im a new user too on ubuntu.but not the first time.im not sure to try this because too many mistake i made since using ubuntu.i will looking for other information about this tutorial on another.egodasahttp://socialmediatoday.com/users/egodasanoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-13384359198299117982014-06-07T03:43:56.012-07:002014-06-07T03:43:56.012-07:00Ok, so I'm trying to make ext4 work as well as...Ok, so I'm trying to make ext4 work as well as possible with SDHC cards and I ran across this. There are a few things I'm wondering though. Firstly, it says "Now turn on expert mode with fdisk and force the partition to begin on 128k alignment. In my case I have set new beginning of data to 256." How do you know exactly what number to put in? Do we just put 256 for anything, or does it vary? And I'm assuming you mean the "b" command in expert mode. Most of us don't exactly use the expert mode options in fdisk very much though (in fact, this is the first time ever that I've seen anything using it.) I'm hoping I can use this to optimize for the Raspberry Pi especially because it so foolishly uses multiple partitions with one being FAT32 and another being Linux-based rather than just one single partition (I guess they chose FAT32 for the sake of being friendlier towards Windows users, but that's kind of silly no matter how I look at it since directly booting Windows on a Raspberry Pi is quite beyond impossible and any Windows users are still going to have to ultimately learn how to use Linux a little bit...) This made it impossible for me to find a way to make the actual Linux partition properly optimized. To this end, I'm guessing when doing the second partition you need to align it as well, but I'm not really sure how to do that (yeah, I don't mess with fdisk much as far as actually aligning things. It's usually just creating and deleting partitions really.)<br /><br />Is this enough though? I originally was trying to figure out how to do what this article describes: http://blogofterje.wordpress.com/2012/01/14/optimizing-fs-on-sd-card/ Unfortunately, the flashbench command just tells me "invalid argument" and I'm beginning to suspect that the -a command was removed within a very very short time after that article was written (the last commits say two years ago, so the project is kind of dead I guess) though oddly the readme still shows using the -a command first thing... (But I'm not 100% sure if it's that or maybe something else. For instance, you'll note that they are using /dev/mmcblk0 which means an integrated card reader system. Unfortunately, you can't really do all this stuff on the Raspberry Pi since it runs from its SD card reader, so I have no clue how to od it that way. Perhaps you have to access the device a certain way for it to work and it won't accept stuff like /dev/sdb though?) Anyway, it seems the point of that article is that the erase boundary could be different depending on the card. Is there any other way we can find out and adapt the partitions to reflect this on a standard PC without a memory card reader built into the motherboard directly (which would likely break legacy compatibility quite a lot anyway)?Nazosanhttps://www.blogger.com/profile/15563547530868384535noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-26378838713768382232014-06-07T03:26:41.127-07:002014-06-07T03:26:41.127-07:00On the subject of FAT32, let's not forget that...On the subject of FAT32, let's not forget that the default format parameters are wrong. In particular it seems you generally need 32K cluster sizes from what I understand, but the best thing to do purportedly is to format it with the Panasonic SD formatter (which of course is Windows only.) Before testing its speeds, try that. As far as aligning things properly goes though, if nothing else it might help decrease the actual number of erases and therefore improve the lifetime of the memory card.<br /><br />BTW, it seems most people feel that turning off journalization is the way to go for the best performance. Though honestly, I think it depends on what you're doing with it. For operations where the data on it is unimportant (say videos thrown onto a card to watch later for instance) performance might be the most important. But for more important stuff the compromise might be necessary. I think this could also be impacting people's results though potentially as it increases the bottleneck somewhat.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-8477531244819933522014-06-07T03:21:58.640-07:002014-06-07T03:21:58.640-07:00For testing with random data, just store the data ...For testing with random data, just store the data in memory (eg a very small ramdrive.) To keep it noob-friendly, I'd say just run from a live CD distro and you're pretty much starting up essentially with a ramdrive right off. You don't need a huge file anyway. Probably just 100MiB is sufficient for performance testing. This eliminates the CPU bottleneck without having a dependency on the host media being fast enough to properly test.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-9169295950409232372014-02-13T10:46:50.773-08:002014-02-13T10:46:50.773-08:00I've tested this out, and found that at least ...I've tested this out, and found that at least with Ubuntu 12.04, following these steps gives no speed increase compared to just creating and formatting the partition with gparted. I also found that the stripe_width parameter to mke2fs didn't make any noticeable different to write speed for me.<br /><br />In fact, I'm not sure that making these changes should even in principle speed up your drive. While it's true that flash drives use large (eg 128k) erase-block sizes, my understanding is that they still write data in sector-sized blocks (512 bytes), and therefore aligning your partition to 128k will not make a difference to write speeds, at least on a fresh drive. It should only make a difference when the drive starts <i>deleting</i> blocks of data, which only happens once the drive gets full. (And even then, partition alignment has very little to do with the speed.)<br /><br />So I don't think that following these steps will actually make your drive any faster. My suspicion is that the only reason the OP found that his drive ran 2x faster than before, is because he changed the filesystem from FAT32 to ext4, and ext4 is a much faster filesystem than FAT. (Or, at a minimum, the linux ext4 driver is faster than the linux VFAT driver.)<br /><br /><br /><br /><br />Postscript: I think I've confirmed this theory. I reformatted with FAT32 on the exact same disk I was testing on before, and found that my write speed went from ~ 7MB/s down to ~ 4.4MB/s. So, moral of the story: if you want it to work faster (at least under Linux), reformat it with ext4 instead of FAT32.Tim Bnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-20393868150718842812014-02-13T09:33:57.563-08:002014-02-13T09:33:57.563-08:00Using /dev/zero to test is actually a bad idea (us...Using /dev/zero to test is actually a bad idea (usually*), as filesystems such as ext4 will store files full of zero using "sparse blocks", meaning that they don't actually write the data to the disk. (Instead, they will just write a small amount of data saying, effectively, "this is a 2GB file full of zeros".) This is why John above observed such a fast write speed with /dev/zero -- because it wasn't actually writing the data.<br /><br />I'd suggest running <b>dd count=100 bs=1M if=/dev/urandom of=/dev/null</b> and checking how fast it goes -- if your drive is slower than this, then you should be right with urandom for input. Otherwise, I'd suggest that the next best bet is to create a 100MB file on a fast storage device (another flash drive?) and use that as the input file, so that you're not generating the randomness on-the-fly.<br /><br /><br />* Of course, if you're using an older/simpler filesystem such as FAT32, that doesn't support sparse blocks, then using /dev/zero is not a problem.Tim Bnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-90899477814731602302014-02-04T14:39:38.752-08:002014-02-04T14:39:38.752-08:00Your guide suggests measuring write speed by writi...Your guide suggests measuring write speed by writing from /dev/urandom However, on a fast drive (e.g. USB3 or if one is testing an internal drive) the random number generator can become a bottleneck. On my machine it maxes out at 14 megs a second. It's best, therefore, to use /dev/zero as the source. e.g.<br /> dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notruncUnknownhttps://www.blogger.com/profile/04280996424692571592noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-35791900983354810992013-11-11T04:39:07.399-08:002013-11-11T04:39:07.399-08:00Hey denu,
No it does not apply to hard disks sinc...Hey denu,<br /><br />No it does not apply to hard disks since they dont have similar erase block technique.Michaelhttps://www.blogger.com/profile/09683047404854369708noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-78435665802668362382013-11-09T07:14:16.060-08:002013-11-09T07:14:16.060-08:00Hi,
Does this apply to external usb hard disks a...Hi, <br /><br />Does this apply to external usb hard disks as well? (like 1TB sized and so on) ?<br />Currently I got 3MB/s write speed and it's actually slower than I can download files from Internet.<br /><br />Best regards<br />denuAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-43453288208173555262013-04-22T07:35:38.434-07:002013-04-22T07:35:38.434-07:00You should really try to use aufs on top of your r...You should really try to use aufs on top of your rootfs in case you're using usb stick for Linux distribution. Try to read this article https://help.ubuntu.com/community/aufsRootFileSystemOnUsbFlash and see if it helps.Michaelhttps://www.blogger.com/profile/09683047404854369708noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-83789928256847477512013-04-09T02:17:39.691-07:002013-04-09T02:17:39.691-07:00I have Kingstone DTSE9 32G usb stick and I've ...I have Kingstone DTSE9 32G usb stick and I've installed an linux operating system on it (Debian Wheezy) ... my question is what I can do to improve writing speed cause is really bad. A simple menu browse will cause system hang and it is simply unusable as hard drive.<br /><br />I try to format like this tutorial said but with no success. I will very appreciate any help or an "step by step" noob tutorial :)Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-87732116507024155972012-11-14T12:41:20.853-08:002012-11-14T12:41:20.853-08:00hi can you help me please ! how can i test read/wr...hi can you help me please ! how can i test read/write speed in linux ? <br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-23251728078262676512012-09-10T07:33:46.752-07:002012-09-10T07:33:46.752-07:00hey John,
try using sync option when mount to mea...hey John,<br /><br />try using sync option when mount to measure real speed.. it seems that in your case you are using OS write cache.Michaelhttps://www.blogger.com/profile/09683047404854369708noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-67254652307396850382012-09-05T01:58:10.508-07:002012-09-05T01:58:10.508-07:00This is insane! Using if=/dev/zero almost writes ...This is insane! Using if=/dev/zero almost writes in an instant!<br /><br />I tried testing this on my Transcend JetFlash 32GB having an ext4 and an xfs filesystem and below are the results. Here are the partition of my Flash Drive.<br /><br />1st Partition - 1GB (ext4) Label: boot-os<br />2nd Partition - 30GB- (xfs) Label: Transcend32G<br /><br />The way I format the 1st partition is:<br /><br /> mkfs.ext4 -E stripe-width=32 -m 0 -O ^has_journal,large_file -L boot-os -vvv /dev/sdc1<br /><br />mke2fs 1.42 (29-Nov-2011)<br />fs_types for mke2fs.conf resolution: 'ext4'<br />warning: 224 blocks unused.<br /><br />Filesystem label=boot-os<br />OS type: Linux<br />Block size=4096 (log=2)<br />Fragment size=4096 (log=2)<br />Stride=0 blocks, Stripe width=32 blocks<br />65664 inodes, 262144 blocks<br />0 blocks (0.00%) reserved for the super user<br />First data block=0<br />Maximum filesystem blocks=268435456<br />8 block groups<br />32768 blocks per group, 32768 fragments per group<br />8208 inodes per group<br />Superblock backups stored on blocks: <br /> 32768, 98304, 163840, 229376<br /><br />Allocating group tables: done <br />Writing inode tables: done <br />Writing superblocks and filesystem accounting information: done<br /><br /><br /><br />The second partition is this:<br /><br />mkfs.xfs -L Transcend32G /dev/sdc2<br /><br />meta-data=/dev/sdc2 isize=256 agcount=4, agsize=1915072 blks<br /> = sectsz=512 attr=2, projid32bit=0<br />data = bsize=4096 blocks=7660288, imaxpct=25<br /> = sunit=0 swidth=0 blks<br />naming =version 2 bsize=4096 ascii-ci=0<br />log =internal log bsize=4096 blocks=3740, version=2<br /> = sectsz=512 sunit=0 blks, lazy-count=1<br />realtime =none extsz=4096 blocks=0, rtextents=0<br /><br /><br /><br />Then I tested their read speed and they're almost identical:<br /><br />N53SV ~ # hdparm -t /dev/sdc1<br /><br />/dev/sdc1:<br /> Timing buffered disk reads: 62 MB in 3.06 seconds = 20.29 MB/sec<br />N53SV ~ # hdparm -t /dev/sdc2<br /><br />/dev/sdc2:<br /> Timing buffered disk reads: 62 MB in 3.06 seconds = 20.24 MB/sec<br /><br /><br /><br />Now here's the fun part using if=/dev/zero<br /><br />john@N53SV:~$ dd count=100 bs=1M if=/dev/zero of=/media/Transcend32G/test.xfs;echo;dd count=100 bs=1M if=/dev/zero of=/media/boot-os/test.ext4<br />100+0 records in<br />100+0 records out<br />104857600 bytes (105 MB) copied, 0.0612922 s, 1.7 GB/s<br /><br />100+0 records in<br />100+0 records out<br />104857600 bytes (105 MB) copied, 0.0589694 s, 1.8 GB/s<br /><br /><br />Using if=/dev/urandom yields the results below:<br /><br />john@N53SV:~$ dd count=100 bs=1M if=/dev/urandom of=/media/Transcend32G/test.xfs;echo;dd count=100 bs=1M if=/dev/urandom of=/media/boot-os/test.ext4<br />100+0 records in<br />100+0 records out<br />104857600 bytes (105 MB) copied, 7.88737 s, 13.3 MB/s<br /><br />100+0 records in<br />100+0 records out<br />104857600 bytes (105 MB) copied, 7.90414 s, 13.3 MB/s<br />JPhttps://www.blogger.com/profile/02455577555650030174noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-31940192780044618072012-08-06T01:01:15.630-07:002012-08-06T01:01:15.630-07:00Great tip. I use Linux to deploy Windows by USB st...Great tip. I use Linux to deploy Windows by USB sticks and this is a good tip. Bookmarked!Ian Atkinhttp://www.symantec.com/connect/user/ianatkinnoreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-86027633323278852222012-07-26T02:12:04.745-07:002012-07-26T02:12:04.745-07:00Are you really serious with this "guide"...Are you really serious with this "guide"? <br />It's really impossible to repartitioning and reformating every USB stick I can use ;) <br />Some of them doesn't belongs to me and I use them to write some data to anyone else... <br />Why the Windows (2k,xp,++) can write 8-10 times faster than Linux? <br />Linux have serious problem with USB sticks - yeah! it is unusable... <br />I had hope that it will be fixed soon, but waiting about 5 years with no advance.<br />Yes i tried all these tricks with ehci and others, sync and async options but still no change. <br />My writing speed on flash disks is about 400kB/s - various HW with linux - the same poor results.<br />Using the same HW with Windows work fine. But any other work in windows is much worse than in linux. <br />Writing to SSD card in my (Samsung) mobile connected via USB is almost instant. Cannot find any difference in mounting parameters. <br />It is much faster to burn data to DVD than copy data to flash disk. That's really poor and sad...Robert Follnerhttps://www.blogger.com/profile/13886867619721782062noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-60546883488625754242012-06-03T22:20:03.564-07:002012-06-03T22:20:03.564-07:00Hi Michael,
my case is a Kingston DT 101 2G 32GB u...Hi Michael,<br />my case is a Kingston DT 101 2G 32GB using NTFS.<br /><br />I called fdisk -H 224 -S 56 /dev/sdd and did the instructions getting <br /><br />Disk /dev/sdd: 31.2 GB, 31221153792 bytes<br />224 heads, 56 sectors/track, 4861 cylinders, total 60978816 sectors<br />Units = sectors of 1 * 512 = 512 bytes<br />Sector size (logical/physical): 512 bytes / 512 bytes<br />I/O size (minimum/optimal): 512 bytes / 512 bytes<br />Disk identifier: 0xc3072e18<br /><br /> Device Boot Start End Blocks Id System<br />/dev/sdd1 256 60978815 30489280 7 HPFS/NTFS/exFAT<br /><br /><br />After (w)riting to save things, I called fdisk /dev/sdd to verify and obtained<br /><br />Disk /dev/sdd: 31.2 GB, 31221153792 bytes<br />44 heads, 24 sectors/track, 57745 cylinders, total 60978816 sectors<br />Units = sectors of 1 * 512 = 512 bytes<br />Sector size (logical/physical): 512 bytes / 512 bytes<br />I/O size (minimum/optimal): 512 bytes / 512 bytes<br />Disk identifier: 0xc3072e18<br /><br /> Device Boot Start End Blocks Id System<br />/dev/sdd1 256 60978815 30489280 7 HPFS/NTFS/exFAT<br /><br />That is, the heads (44) and sectors (24) have changed !!! <br />Tried several times and the results were the same.<br /><br />Is this the expected behaviour? This also happened with you?<br /><br /><br />As I need to use NTFS (ntfs-3g), what the necessary parameters while making the file system<br />with mkntfs to obtain the correct alignment?<br /><br />Regards<br />FabioFábio Freitashttps://www.blogger.com/profile/03514804749489327738noreply@blogger.comtag:blogger.com,1999:blog-8511499809446054798.post-44717400811419713072012-05-14T01:39:28.730-07:002012-05-14T01:39:28.730-07:00Interesting, but confusing (for a noob like me...)...Interesting, but confusing (for a noob like me...)<br /><br />1. For the read-test you use "/dev/sdb" and for the write-test "/media/disk/test". Am I correct that this should also be /dev/sdb ?<br />2. Shouldn't people be warned NOT to run dd on a stick containing data!?!?<br />3. Can you explain a bit more about the fdisk/expert/begin partition please? Do I first create the partition and then hit 'x' for expert and then change to..... what? My default states 2048, do I change that to 256? Is that the same for every size stick?<br />4. I tried this and the outcom is exactly the same speed.Edhttps://www.blogger.com/profile/15004896525342623451noreply@blogger.com