16

I have seen many posts about creating a unique filename from the naive %TIME% to the plausible (but insufficient) %RANDOM%. Using wmic os get localdatetime is much better, but it can still fail on multiple CPU/core machines. The following script will eventually fail when run in 5+ shells on a multple core machine.

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

FOR /L %%i IN (0, 1, 1000) DO (
    FOR /F "usebackq" %%x IN (`wmic os get localdatetime ^| find "."`) do (set MYDATE=%%x)
    ECHO MYDATE is now !MYDATE!
    IF EXIST testuniq_!MYDATE!.txt (
        ECHO FAILED ON !MYDATE!
        GOTO TheEnd
    )
    COPY NUL >testuniq_!MYDATE!.txt
)

:TheEnd
EXIT /B 0

Does anyone have a reliable way to create a unique file name in a shell script?

4
  • 2
    It's not often we get a question in batch-file land that warrants over 5 answers. +1 interesting question! Commented Jan 6, 2015 at 21:50
  • You could always use %RANDOM%%RANDOM% to make a longer, less likely number(?) Commented Jul 21, 2023 at 14:17
  • Step 1: Generate a unique ID. See stackoverflow.com/questions/4313422/… Step 2: Use the unique ID as filename Commented Aug 21, 2023 at 10:51
  • @JohanWitters, there are even stronger answers than using a GUID? Commented Aug 21, 2023 at 11:34

13 Answers 13

13

Any system that relies on a random number can theoretically fail, even one using a GUID. But the world seems to have accepted that GUIDs are good enough. I've seen code posted somewhere that uses CSCRIPT JScript to generate a GUID.

There are other ways:

First I will assume that you are trying to create a unique temporary file. Since the file will be deleted once your process ends, all you must do is establish an exclusive lock on a resource whose name is a derivative of your temp file name. (actually I go in reverse - the temp name is derived from the locked resource name).

Redirection establishes an exclusive lock on the output file, so I simply derive a name from the time (with 0.01 second preciscion), and attempt to lock a file with that name in the user's temp folder. If that fails than I loop back and try again until I succeed. Once I have success, I am guaranteed to have sole ownership of that lock, and all derivitives (unless someone intentionally breaks the system).

Once my process terminates, the lock will be released, and a temp file with the same name could be reused later on. But the script normally deletes the temp file upon termination.

@echo off
setlocal

:getTemp
set "lockFile=%temp%\%~nx0_%time::=.%.lock"
set "tempFile=%lockFile%.temp"
9>&2 2>nul (2>&9 8>"%lockFile%" call :start %*) || goto :getTemp

:: Cleanup
2>nul del "%lockFile%" "%tempFile%"
exit /b


:start
:: Your code that needs the temp file goes here. This routine is called with the
:: original parameters that were passed to the script. I'll simply write some
:: data to the temp file and then TYPE the result.
>"%tempFile%" echo The unique tempfile for this process is "%tempfile%"
>>"%tempFile%" echo(%*
type "%tempFile%"
exit /b

Looping due to name collision should be rare unless you are really stressing your system. If so, you can reduce the chance of looping by a factor of 10 if you use WMIC OS GET LOCALDATETIME instead of %TIME%.


If you are looking for a persistent unique name, then the problem is a bit more difficult, since you cannot maintain the lock indefinitely. For this case I recommend the WMIC OS LocalDateTime approach, coupled with two checks for name collision.

The first check simply verifies the file does not already exist. But this is a race condition - two processes could make the check at the same time. The second check creates the file (empty) and establishes a temporary exclusive lock on it. The trick is to make sure that the lock is maintained for a period of time that is longer than it takes for another process to check if the file exists. I'm lazy, so I simply use TIMEOUT to establish a 1 second wait - way more than should be necessary.

The :getUniqueFile routine expects three arguments - a base name, an extension, and the variable name where the result is to be stored. The base name can include drive and path information. Any path information must be valid, otherwise the routine will enter an infinite loop. That issue could be fixed.

@echo off
setlocal

:: Example usage
call :getUniqueFile "d:\test\myFile" ".txt" myFile
echo myFile="%myFile%"
exit /b

:getUniqueFile  baseName  extension  rtnVar
setlocal
:getUniqueFileLoop
for /f "skip=1" %%A in ('wmic os get localDateTime') do for %%B in (%%A) do set "rtn=%~1_%%B%~2"
if exist "%rtn%" (
  goto :getUniqueFileLoop
) else (
  2>nul >nul (9>"%rtn%" timeout /nobreak 1) || goto :getUniqueFileLoop
)
endlocal & set "%~3=%rtn%"
exit /b

The above should be guaranteed to return a new unique file name for the given path. There is a lot of room for optimization to establish some command to execute during the lock check that takes "long enough" but not "too long"

Sign up to request clarification or add additional context in comments.

14 Comments

Your lock file idea seems to be the most foolproof solution. Forcing each instance to establish an exclusive lock guarantees that no two concurrent instances will ever be able to use the same file. It's immune to race conditions. If I were the O.P. this would win.
I agree. This seems to be the most industrial strength solution.
@dbenham - I am revisiting this topic. The problem is that my "Your code" needs to return an exit code. If it returns a non-zero value, control transfers to the loop to get a unique name.
@dbenham - This is working well for me. I had to add one line to remove any space characters because the TIME format might be a 12-hour clock with a leading space. set "lockFile=%lockFile: =%
@dbenham Actually, only the second method fails to work with stdin: script.cmd < input.txt causes it to enter an infinite loop. But maybe if you can reproduce it, you will also be able to fix it for this use case.
|
8

You could use certutil to base64 encode %date% %time% with a %random% seed like this:

@echo off
setlocal

:: generate unique ID string
>"%temp%\~%~n0.%username%.a" echo %username%%date%%time%%random%
>NUL certutil -encode "%temp%\~%~n0.%username%.a" "%temp%\~%~n0.%username%.b"
for /f "usebackq EOL=- delims==" %%I in ("%temp%\~%~n0.%username%.b") do set "unique_id=%%I"
del "%temp%\~%~n0.%username%.a" "%temp%\~%~n0.%username%.b"

echo %unique_id%

In case the same script is being run from the same directory by multiple users, I added %username% to the temp files to avoid further conflict. I suppose you could replace %random% with %username% for the same effect. Then you'd only get a conflict if a single user executes the same code block twice concurrently.

(Edit: added %username% as a seed for uniqueness.)

2 Comments

Will certutil always produce the same output it the input it the same? If so, then if the original text, %date%%time%%random%, is not unique, then encoding it is not going to help. In my experimentation, the original text is not always unique.
How many threads are likely to run within the same centisecond by the same user? You could add %username% as a seed as well. Or if you want to increase the randomness of %random%, then set /a "rand = (%random% + 1) * (%random% + 1) * %random%" should result in a random integer in the full signed 32-bit range of -2147483648 to 2147483647. (Because when a number exceeds 32-bits, the cmd interpreter rolls over to negative numbers.) True that there's some weighting and this won't result in derivatives of prime numbers, but you're still getting more randomness nevertheless.
4

The Batch-JScript hybrid script below uses WSH's fso.GetTempName() method that was designed precisely for this purpose:

@if (@CodeSection == @Batch) @then

@echo off

for /F "delims=" %%a in ('cscript //nologo //E:JScript "%~F0"') do set "fileName=%%a"
echo Created file: "%fileName%"
goto :EOF

@end

var fso = new ActiveXObject("Scripting.FileSystemObject"), fileName;
do { fileName = fso.GetTempName(); } while ( fso.FileExists(fileName) );
fso.CreateTextFile(fileName).Close();
WScript.Echo(fileName);

5 Comments

This is looking pretty good. I will probably choose this as the answer after some more testing in integration into a larger project.
At first I thought that the name generated by GetTempName() was guaranteed to be unique. But based on my reading of various unofficial sources, it appears that the returned name is simply a random number, without any true verification that it is unique. I think you would be better of using JScript to generate a GUID.
@dbenham: It doesn't matter if GetTempName() generate an unique name or not because the additional FileExists(fileName) test guarantee that the file is unique. GetTempName() is just a method simpler to use than the combination of random/time values.
But you have a race condition. Two processes could make the check with the same name at the same time, and both would succeed and return duplicate names. (Unlikely, but it is possible)
Re: using JScript to generate a GUID, I added a simpler powershell example already. But I've got to say, although it pains me to post 2 solutions while recommending someone else's, I think dbenham's solution is the most bullet-proof. Damn it.
4

I like Aacini's JScript hybrid solution. As long as we're borrowing from other runtime environments, how about using .NET's System.GUID with PowerShell?

@echo off
setlocal

for /f "delims=" %%I in ('powershell -command "[string][guid]::NewGuid()"') do (
    set "unique_id=%%I"
)

echo %unique_id%

Comments

3
@echo off
:: generate a tempfilename in %1 (default TEMPFILE)
:loop
set /a y$$=%random%+100000
set y$$=temp%y$$:~1,2%.%y$$:~-3%
if exist "%temp%\%y$$%" goto loop
SET "y$$=%temp%\%y$$%"&copy nul "%temp%\%y$$%" >nul 2>NUL
:: y$$ now has full tempfile name
if "%1"=="" (set "tempfile=%y$$%") else (set "%1=%y$$%")

Here's a cut-down version of my tempfile generator to create a file named tempnn.nnn in the %temp% directory.

Once tempnn.nnn has been created, then it's simple to create as many further tempfiles as you like for the process by appending a suffix to %y$$%, eg %y$$%.a etc. Of course, that presumes that some other process doesn't randomly create filenames without using this procedure.

4 Comments

I like that your method makes damn sure the file doesn't exist with if exist and a loop.
I believe there are two issues here. First, %random% is often not random. Secondly, the time between the test (if exist) and the set (copy nul) leaves an opportunity for failure.
@Paul: The looping on file-exists takes care of the rndom-isn't-random issue. Certainly, there is a theoretical possibility that a file may appear between the test and copy, but there's no way batch can deal with that. It can't create and then test since it may then kill a valid file created by another process, so we're left with the test-and-create method. Limitation of the language - and unless you can guarantee that your process isn't interrupted, a limitation of any language.
I agree with Paul. See my answer for a fool proof way to "deal with that".
1

A long time ago in a galax..newsgroup called alt.msdos.batch, a contributor insisted on posting a QBASIC solution to each issue raised, wrapped in a batch shell and claiming it was batch because it only used standard Microsoft-supplied software.

After a long time, he was cured of his errant ways, but I've been severely allergic to hybrid methods ever since. Sure - if it cures the problem, blah,blah - and sometimes there's no escaping that course (run batch silently, for instance.) Apart from that, I really don't want thebother of learning a new language or two...So that's why I eschew *script solutions. YMMV.

So - here's another approach...

@ECHO OFF
SETLOCAL
:rndtitle
SET "progtitle=My Batch File x%random%x"
TITLE "%progtitle%"
SET "underline="
SET "targetline="
FOR /f "delims=" %%a IN ('TASKLIST /v^|findstr /i /L /c:"======" /c:"%progtitle%"') DO (
 IF DEFINED underline (
  IF DEFINED targetline GOTO rndtitle
  SET "targetline=%%a"
 ) ELSE (SET "underline=%%a")
)

:: Here be a trap. The first column expands to fit the longest image name...
:pidloop
IF "%underline:~0,1%"=="=" SET "underline=%underline:~1%"&SET "targetline=%targetline:~1%"&GOTO pidloop
FOR /f %%a IN ("%targetline%") DO SET /a pid=%%a
ECHO %pid%

GOTO :EOF

Essentially, set the title to something random. Note that x%random%x is used, not simply %random% so that a search for "x123x" won't detect "x1234x".

Next, get a verbose tasklist, locating the line underlining the heading and the title. The first line returned by findstr will be the underline, the next will set targetline and any further returns indicate that the title is not unique, so go back, change it and try again until it is unique.

Finally, get the process ID which is in the second column, noting that the first is of unknown width (hence why the underline is grabbed.)

Result : this process's PID which must be unique and hence can be used as the basis for a unique tempfile name - build it in a directory reserved for the purpose.

1 Comment

While I did try to get the PID before, I never came to a workable solution. Screen scraping the output of tasklist.exe looks like it will work. But, I think Powershell might be easier and more direct.
1

you can try with guid.bat

for /f "tokens=* delims={}" %%# in ('guid.bat') do set "unique=%%#"
echo %unique%

for usage directly from console:

for /f "tokens=* delims={}" %# in ('guid.bat') do @set "unique=%#"

Comments

1

I would like to submit another method and find out if there are any holes is it. Please let me know. Thanks.

C:>title my shell a80en333xyb

C:>type getppid.ps1
(Get-WmiObject -Class Win32_Process -Filter "processid='$pid'").ParentProcessId

C:>powershell getppid.ps1
2380

Or, to run it without creating a .ps1 file:

powershell -NoProfile -Command "(Get-CimInstance -ClassName Win32_Process -Filter "ProcessId=`'$pid`'").ParentProcessId"

To verify that the correct task is found, the title is set to an unlikely value and tasklist.exe is used to search for the value returned by getppid.ps1.

C:>tasklist /v | find "2380"
cmd.exe                       2380 RDP-Tcp#0                  7      8,124 K Running         PHSNT\pwatson                                           0:00:03 my shell a80en333xyb

Comments

0

make the file contents an object, use the pointer memaddress as the first part of your file name and a Rand() as your second part. the memory address will be unique for all objects even with multiple instances running.

5 Comments

Nice ideas if I was using a language such as C/C++, Python, Perl, Java, etc. This is a DOS/Windows batch file script. I can write those other things, but trying not to.
I don't see how this could be guaranteed to be unique if there are multiple machines writing to a shared device.
if you are using a batch script then perhaps "machine name - process ID - Timestamp - random number" should be sufficiently unique.
This will only work for temporary files, as long as the memory is reserved throughout the life of the temporary file. True, no other process could have that mem address at a given point in time, but the same address could be returned at different points in time for unrelated processes, leading to name collision if the file is persistent (not temporary).
true. OK to be certain of universal uniqueness accross mutiple machines, multiple processes per machine with persistant filenames, you would proably need to go: MachineName (or better: GUID) /ProcessID/TimeStamp/InternalMEMaddress and you can stick a RAND() on the end if you want, but it'd probably be overkill. Also I'm pretty sure you are going to hit the max filename size on windows systems with that.but truncating the memaddress to least significat 8 digits should be enough alongside the timestamp and PID/UID unless your files are very large and you are generating hundreds per second.
0

At first some remarks for using a PID as a part of a unique file name:

When you just look for a parent PID this might be ambiguous because when you run that code in a for loop or in a call within your batch file always a new cmd.exe is created and you get the PID of this temporary cmd.exe process.

So you should rather use the PID of your batch "root" cmd.exe process. In order to get that - instead of searching by title in the the task list, which is slow and might also be ambiguous, there is another approach.

You can get the PID of your batch "root" cmd.exe process by using Windows API functions:

  • GetConsoleWindow provides the console window handle
  • GetWindowThreadProcessId provides the PID by using this console window handle

You can run these Windows API functions by using PowerShell:

Add-Type -MemberDefinition @"
  // HWND WINAPI GetConsoleWindow(void)
  [DllImport("kernel32.dll", EntryPoint = "GetConsoleWindow")]
  public static extern IntPtr GetConsoleWindow();

  // DWORD GetWindowThreadProcessId(HWND hWnd, LPDWORD lpdwProcessId)
  [DllImport("user32.dll", EntryPoint = "GetWindowThreadProcessId")]
  public static extern uint GetWindowThreadProcessId(IntPtr hWnd, out uint lpdwProcessId);

  public static uint GetPID()
  {
    IntPtr h = GetConsoleWindow();
    if ((uint)h==0) return 0;
    uint rc = 0;
    GetWindowThreadProcessId(h, out rc);
    return rc;
  }
"@ -Name Win32 -NameSpace System

[System.Win32]::GetPID();

Within your batch file you can run the PowerShell code as an encoded command:

set MyPID=
for /f %%a in ('powershell -NoLogo -NoProfile -NonInteractive -EncodedCommand QQBkAGQALQBUAHkAcABlACAALQBNAGUAbQBiAGUAcgBEAGUAZgBpAG4AaQB0AGkAbwBuACAAQAAiAA0ACgBbAEQAbABsAEkAbQBwAG8AcgB0ACgAIgBrAGUAcgBuAGUAbAAzADIALgBkAGwAbAAiACwAIABFAG4AdAByAHkAUABvAGkAbgB0ACAAPQAgACIARwBlAHQAQwBvAG4AcwBvAGwAZQBXAGkAbgBkAG8AdwAiACkAXQANAAoAcAB1AGIAbABpAGMAIABzAHQAYQB0AGkAYwAgAGUAeAB0AGUAcgBuACAASQBuAHQAUAB0AHIAIABHAGUAdABDAG8AbgBzAG8AbABlAFcAaQBuAGQAbwB3ACgAKQA7AA0ACgBbAEQAbABsAEkAbQBwAG8AcgB0ACgAIgB1AHMAZQByADMAMgAuAGQAbABsACIALAAgAEUAbgB0AHIAeQBQAG8AaQBuAHQAIAA9ACAAIgBHAGUAdABXAGkAbgBkAG8AdwBUAGgAcgBlAGEAZABQAHIAbwBjAGUAcwBzAEkAZAAiACkAXQANAAoAcAB1AGIAbABpAGMAIABzAHQAYQB0AGkAYwAgAGUAeAB0AGUAcgBuACAAdQBpAG4AdAAgAEcAZQB0AFcAaQBuAGQAbwB3AFQAaAByAGUAYQBkAFAAcgBvAGMAZQBzAHMASQBkACgASQBuAHQAUAB0AHIAIABoAFcAbgBkACwAIABvAHUAdAAgAHUAaQBuAHQAIABsAHAAZAB3AFAAcgBvAGMAZQBzAHMASQBkACkAOwANAAoAcAB1AGIAbABpAGMAIABzAHQAYQB0AGkAYwAgAHUAaQBuAHQAIABHAGUAdABQAEkARAAoACkADQAKAHsADQAKACAASQBuAHQAUAB0AHIAIABoACAAPQAgAEcAZQB0AEMAbwBuAHMAbwBsAGUAVwBpAG4AZABvAHcAKAApADsADQAKACAAaQBmACAAKAAoAHUAaQBuAHQAKQBoAD0APQAwACkAIAByAGUAdAB1AHIAbgAgADAAOwANAAoAIAB1AGkAbgB0ACAAcgBjACAAPQAgADAAOwANAAoAIABHAGUAdABXAGkAbgBkAG8AdwBUAGgAcgBlAGEAZABQAHIAbwBjAGUAcwBzAEkAZAAoAGgALAAgAG8AdQB0ACAAcgBjACkAOwANAAoAIAByAGUAdAB1AHIAbgAgAHIAYwA7AA0ACgB9AA0ACgAiAEAAIAAtAE4AYQBtAGUAIABXAGkAbgAzADIAIAAtAE4AYQBtAGUAUwBwAGEAYwBlACAAUwB5AHMAdABlAG0ADQAKAFsAUwB5AHMAdABlAG0ALgBXAGkAbgAzADIAXQA6ADoARwBlAHQAUABJAEQAKAApADsADQAKAA^=^= 2^>nul') do (
  set "MyPID=%%~a"
)

if defined MyPID echo %MyPID%

Comments

0

generate unique number (datetime stamp)

@echo off
for /f "tokens=2 delims==" %%a in ('wmic OS Get localdatetime /value') do set "dt=%%a"
set "YY=%dt:~2,2%" & set "YYYY=%dt:~0,4%" & set "MM=%dt:~4,2%" & set "DD=%dt:~6,2%"
set "HH=%dt:~8,2%" & set "Min=%dt:~10,2%" & set "Sec=%dt:~12,2%"
rem set "datestamp=%YY%%MM%%DD%" & set "timestamp=%HH%%Min%%Sec%"
set "datestamp=%YYYY%%MM%%DD%" 
set "timestamp=%HH%%Min%%Sec%"
set unique_number=%datestamp%%timestamp%
echo %unique_number%

Comments

0

Here is another way. Note that the newly created file will be created with zero (0) length. Your code is responsible for removing it if needed.

C:>TYPE CreateTempFile.bat
FOR /F "delims=" %%A IN ('powershell.exe -NoLogo -NoProfile -Command "(New-TemporaryFile).FullName"') DO (SET "NEWFILE=%%~A")
ECHO New file is "%NEWFILE%"
DIR "%NEWFILE%"
IF EXIST "%NEWFILE%" (DEL "%NEWFILE%")

Comments

0

I think, we can significantly decrease the complexity of script by starting with a simple way to generate the temp file:

prompt>set tempfile=%tmp%\%random%
prompt>echo %tempfile%
C:\templocation\Temp\30912

Ok, there is still a small probability of 1/2^15=1/32768 that generated name could clash with some existing file name, only if other programs uses the same file name generating algorithm. Keeping in mind, different programs uses different algorithms of file names generation, this probability will be generally lower by orders of magnitudes.
Anyway, we can increase the paranoia level in a very trivial way, without increasing the complexity of the script. Randomizing 4 times, the probability decreases exponentially by a factor of 4, to (1/2^15)^4=1/1.1e18. So, if your program generates a billion of file names, the probability of clashes will be still 1/billion.

prompt>set tempfile=%tmp%\%random%%random%%random%%random%
prompt>echo %tempfile#
C:\templocation\Temp\3072821772196416436

Also, consider adding project specific suffixes, prefixes, extensions ... This way, the name clashes can occur only between files generated by your project. This will reduce the duplicates probability to zero by any possible practical meaning:

prompt>set tempfile=%tmp%\buzz.%random%%random%%random%%random%.bar.neotxt
prompt>echo %tempfile#
C:\templocation\Temp\buzz.1403156481473423009.bar.neotxt

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.