this.title = "gg Microsoft"
this.metadata = {
rant: true,
long: true,
super_long: true,
has_summary: true
// Also:
let microsoft = "dead" // please?

tl;dr: Windows' MAX_PATH is the devil, and it basically does not allow you to copy files with paths that exceed this length. No matter what. Even with official fixes and workarounds.

Long story:
So, I haven't had actual gainful employ in quite awhile. I've been earning just enough to get behind on bills and go without all but basic groceries. Because of this, our electronics have been ... in need of upgrading for quite awhile. In particular, we've needed new drives. (We've been down a server for two years now because its drive died!)

Anyway, I originally bought my external drive just for backup, but due to the above, I eventually began using it for everyday things. including Steam. over USB. Terrible, right? So, I decided to mount it as an internal drive to lower the read/write times. Finding SATA cables was difficult, the motherboard's SATA plugs are in a terrible spot, and my tiny case (and 2yo) made everything soo much worse. It was a miserable experience, but I finally got it installed.

However! It turns out the Seagate external drives use some custom drive header, or custom driver to access the drive, so Windows couldn't read the bare drive. ffs. So, I took it out again (joy) and put it back in the enclosure, and began copying the files off.

The drive I'm copying it to is smaller, so I enabled compression to allow storing a bit more of the data, and excluded a couple of directories so I could copy those elsewhere. I (barely) managed to fit everything with some pretty tight shuffling.

but. that external drive is connected via USB, remember? and for some reason, even over USB3, I was only getting ~20mb/s transfer rate, so the process took 20some hours! In the interim, I worked on some projects, watched netflix, etc., then locked my computer, and went to bed. (I also made sure to turn my monitors and keyboard light off so it wouldn't be enticing to my 2yo.) Cue dramatic music ~

Come morning, I go to check on the progress... and find that the computer is off! What the hell! I turn it on and check the logs... and found that it lost power around 9:16am. aslkjdfhaslkjashdasfjhasd. My 2yo had apparently been playing with the power strip and its enticing glowing red on/off switch. So. It didn't finish copying.

aslkjdfhaslkjashdasfjhasd x2

Anyway, finding the missing files was easy, but what about any that didn't finish? Filesizes don't match, so writing a script to check doesn't work. and using a visual utility like windirstat won't work either because of the excluded folders. Friggin' hell.

Also -- and rather the point of this rant:

It turns out that some of the files (70 in total, as I eventually found out) have paths exceeding Windows' MAX_PATH length (260 chars). So I couldn't copy those.

After some research, I learned that there's a Microsoft hotfix that patches this specific issue! for my specific version! woo! It's like. totally perfect. So, I installed that, restarted as per its wishes... tried again (via both drag and `copy`)... and Lo! It did not work.

After installing the hotfix. to fix this specific issue. on my specific os. the issue remained. gg Microsoft?

Further research.

I then learned (well, learned more about) the unicode path prefix `\\?\`, which bypasses Windows kernel's path parsing, and passes the path directly to ntfslib, thereby indirectly allowing ~32k path lengths. I tried this with the native `copy` command; no luck. I tried this with `robocopy` and cygwin's `cp`; they likewise failed. I tried it with cygwin's `rsync`, but it sees `\\?\` as denoting a remote path, and therefore fails.

However, `dir \\?\C:\` works just fine?

So, apparently, Microsoft's own workaround for long pathnames doesn't work with its own utilities. unless the paths are shorter than MAX_PATH? gg Microsoft.

At this point, I was sorely tempted to write my own copy utility that calls the internal Windows APIs that support unicode paths. but as I lack a C compiler, and haven't coded in C in like 15 years, I figured I'd try a few last desperate ideas first.

For the hell of it, I tried making an archive of the offending files with winRAR. Unsurprisingly, it failed to access the files.

... and for completeness's sake -- mostly to say I tried it -- I did the same with 7zip. I took one of the offending files and made a 7z archive of it in the destination folder -- and, much to my surprise, it worked perfectly! I could even extract the file! Hell, I could even work with paths >340 characters!

So... I'm going through all of the 70 missing files and copying them. with 7zip. because it's the only bloody thing that works. ffs

Third-party utilities work better than Microsoft's official fixes. gg.

On a related note, I totally feel like that person from http://xkcd.com/763 right now ;;

  • 10
    I've recently discovered robocopy is installed on Windows by default and is actually pretty smart. If you haven't heard of it, I recommend looking into it.

    My favorite is it will copy everything it can w/o error, and then everything else it will try again every 30 seconds up to a year.
  • 0
    @paco I specifically tried robocopy; no luck.
    It is a rather nice utility, though!
  • 1
    @Letmecode ^_^ Thanks!
  • 5
    Ahhh I totally forgot to mention that PowerShell doesn't support Unicode paths either!
  • 4
    I use Total Commander. It uses the new API and handles long paths just fine, which is actually sometimes a drawback because it can create files other applications can't access.
  • 1
    Would it have worked if you shortened the parent folder names? o.O
  • 3
    I love rants like this, shows how even the simplest premise in the tech world can snowball.
  • 3
    @neriald Yes, but renaming enough folders would have been quite tedious, and destructive in two instances.
  • 2
    I found that gitbash is very good at handling that kind of file system shit in Windows.
  • 1
    @runfrodorun they congratulated me when I shared a detailed problem of the error code I was getting, even gave me a £50 coupon, don't know where to use it though....
  • 1
    This might be just one case where Linux comes in handy. I mostly use Windows, but when I want to make a copy of, say, the entire hard drive, I've found that it's much less hassle to do it with Linux.
  • 3
    @TerriToniAX I've found Linux to be much less of a hassle in general.
  • 0
    I guess you didn't try to copy the files with The Ubuntu subsystem?
  • 0
    @olback Windows 7.
  • 0
    @Ashkin Aah.
  • 2
    @Ashkin Nah, I don't agree. In this particular case Linux is better, but in many aspects Windows is less hassle. Being targeted to "common people" (= lamers/non-devs) as it is, you don't need to code your own ethernet drivers or even compile any stuff that you would just run an installer for in Windows. And when running into a problem, it is much easier to find help for Windows at different user fora etc. When someone has asked the same support question in a Linux user forum, it would render an answer like "Are you a stupid lamer or what?". Well not those exact words, but the message being the same, if you're not an expert then leave this fine OS and its superior users alone.
  • 0
    @TerriToniAX You don't say.
  • 1
    @Jop- Last was a couple of months ago when I tried to install Linux Mint Cinnamon on an old Acer laptop and get a driver for its BroadCom WLAN adapter. I was unable to find any kind of installer, only source code that had to be compiled. Turned out though that the adaptor was broken anyway, didn't work with Windows neither.
  • 1
    @Jop- OK, maybe it didn't work because the adaptor was broken then. In that case, it isn't Linux's fault at all :)
  • 0
    Hi, one thing that I could suggest is, if you have long file names robocopy" error, you should try to use a LongPath tool. It can be the best solution.
  • 0
    @Root Sure, but the freeness can have it's downsides. E.g. wasn't easy to rm files with special characters in it (the horror of bash escape sequences), needed to google it several times. Another problem is when shell expansion breaks down because of too many files present. But luckily there are usually workarounds
Add Comment