asynchronous file operations

[DirectuX]

Hi,

thinBasic\SampleScripts\UI\Power\Power_Messages.tBasic is a sample that can detect power status changes via callback management.

I'm in search of such behaviour for complete filesystem interaction.

As a start I studied the file copy feature:

Both FILE_Copy and FILE_ShellCopy are synchronous functions. None can expose the copy progression to the script.

Instead, MS documentation shows two relevant entries: IFileOperation interface and CopyFileExW function.

Before moving in any direction, I would like to ask :

• advices for which of these two last way-suggestions would be the fittest and convenient to implement ?
• in your opinion, which shape the implementation should take ?
• if you think of another way ?
• any other thought about this ?

[DirectuX]

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

[ErosOlmi]

This is an example on how to use shell file operations.
Hope can be of some help.

References: https://docs.microsoft.com/en-us/win...fileoperationa

Following example just copy script itself into a copy with extension .TXT

'--------------------------------------------------------------------
' https://docs.microsoft.com/en-us/windows/win32/api/shellapi/nf-shellapi-shfileoperationa
'--------------------------------------------------------------------




'---Shell File Operations
  %FO_MOVE                   = &H0001
  %FO_COPY                   = &H0002
  %FO_DELETE                 = &H0003
  %FO_RENAME                 = &H0004


'---Shell File Options
  %FOF_MULTIDESTFILES        = &H0001
  %FOF_CONFIRMMOUSE          = &H0002
  %FOF_SILENT                = &H0004  ' don't display progress UI (confirm prompts may be displayed still)
  %FOF_RENAMEONCOLLISION     = &H0008  ' automatically rename the source files to avoid the collisions
  %FOF_NOCONFIRMATION        = &H0010  ' don't display confirmation UI, assume "yes" for cases that can be bypassed, "no" for those that can not
  %FOF_WANTMAPPINGHANDLE     = &H0020  ' Fill in SHFILEOPSTRUCT.hNameMappings
                                             ' Must be freed using SHFreeNameMappings
  %FOF_ALLOWUNDO             = &H0040  ' enable undo including Recycle behavior for IFileOperation::Delete()
  %FOF_FILESONLY             = &H0080  ' only operate on the files (non folders), both files and folders are assumed without this
  %FOF_SIMPLEPROGRESS        = &H0100  ' means don't show names of files
  %FOF_NOCONFIRMMKDIR        = &H0200  ' don't dispplay confirmatino UI before making any needed directories, assume "Yes" in these cases
  %FOF_NOERRORUI             = &H0400  ' don't put up error UI, other UI may be displayed, progress, confirmations
  %FOF_NOCOPYSECURITYATTRIBS = &H0800  ' dont copy file security attributes (ACLs)
  %FOF_NORECURSION           = &H1000  ' don't recurse into directories for operations that would recurse
  %FOF_NO_CONNECTED_ELEMENTS = &H2000  ' don't operate on connected elements ("xxx_files" folders that go with .htm files)
  %FOF_WANTNUKEWARNING       = &H4000  ' during delete operation, warn if nuking instead of recycling (partially overrides FOF_NOCONFIRMATION)
  %FOF_NORECURSEREPARSE      = &H8000?? ' deprecated; the operations engine always does the right thing on FolderLink objects (symlinks, reparse points, folder shortcuts)
  %FOF_NO_UI = (%FOF_SILENT OR %FOF_NOCONFIRMATION OR %FOF_NOERRORUI OR %FOF_NOCONFIRMMKDIR)  ' don't display any UI at all




TYPE SHFILEOPSTRUCTA
    hwnd                  AS DWORD         ' HWND
    wFunc                 AS DWORD
    pFrom                 AS ASCIIZ PTR
    pTo                   AS ASCIIZ PTR
    fFlags                AS Word
    fAnyOperationsAborted AS LONG          ' BOOL
    hNameMappings         AS DWORD         ' LPVOID
    lpszProgressTitle     AS ASCIIZ PTR    ' only used if FOF_SIMPLEPROGRESS
END TYPE


DECLARE FUNCTION SHFileOperationA LIB "Shell32.dll" ALIAS "SHFileOperationA" (byref lpFileOp AS SHFILEOPSTRUCTA) AS LONG


'--------------------------------------------------------------------
' Copy file routine, using shell function (with dialog)
'--------------------------------------------------------------------
FUNCTION ShellFileCopy(BYVAL Source AS STRING, BYVAL Destination AS STRING) AS LONG
  Local lRes  As Long
  Local shfos As SHFILEOPSTRUCTA


  Source        = Source + Chr$(0) + Chr$(0)
  Destination   = Destination + Chr$(0) + Chr$(0)


  shfos.wFunc   = %FO_COPY
  shfos.pFrom   = StrPtr(Source)
  shfos.pTo     = StrPtr(Destination)
  shfos.fFlags  = %FOF_NOCONFIRMMKDIR Or %FOF_MULTIDESTFILES


  'Note: if to use FOF_MULTIDESTFILES, source and destination strings can
  'contain multiple files, separated with CHR$(0), and end with CHR$(0,0)


  lRes = SHFileOperationA(shfos)


  If lRes <> %Null Or shfos.fAnyOperationsAborted <> 0 Then
     'user aborted, do whatever is needed..
  END IF


  FUNCTION = lRes
END FUNCTION


uses "Console"


string sFileSource      = APP_ScriptFullName
string sFileDestination = APP_ScriptFullName & ".copy.txt"
long   lRet


printl "Copying: " & sFileSource
printl "To.....: " & sFileDestination


lRet = ShellFileCopy(sFileSource, sFileDestination)


printl "Result.: " & lret


printl "All done. Press a key to end"
WaitKey

[DirectuX]

Thanks you Eros for the sample.
I tried it, but this does the same as thinbasic's FILE_ShellCopy function (synchronous).

That is why I proposed two other functions that (to my knowledge) would allow the same behaviour than for thinBasic\SampleScripts\UI\Power\Power_Messages.tBasic (callbacks).

What do you think about it ?

[ErosOlmi]

I think the API to use is CopyFileExA
https://docs.microsoft.com/en-us/win...se-copyfileexa

Using a CODEPTR to a lpProgressRoutine
https://docs.microsoft.com/it-it/win...ogress_routine
Example using PowerBasic language: https://forum.powerbasic.com/forum/u...ine#post782798

Problem is that thinBasic allow CODEPTR only to functions with a maximum of 4 parameters while in this case you need 9 parameters.
I think the only way is to develop a compiled FILE module function that wrap CopyFileEx allowing passing a script function that will be called when needed.

Will see what I can do but not just round the corner.

[DirectuX]

I think the API to use is CopyFileExA
https://docs.microsoft.com/en-us/win...se-copyfileexa

The W is for unicode. Some filename are foreign.

Using a CODEPTR to a lpProgressRoutine
https://docs.microsoft.com/it-it/win...ogress_routine
Example using PowerBasic language: https://forum.powerbasic.com/forum/u...ine#post782798

Problem is that thinBasic allow CODEPTR only to functions with a maximum of 4 parameters while in this case you need 9 parameters.

Thought it would be easier
I see LPPROGRESS_ROUTINE callback function
Can it be received by example in a string or an udt that I cut after ? (aka 1 parameter ?) <- or my logic is wrong ? Edit1: From tb help: script function can have from zero to 4 BYVAL LONG or DWORD parameters

I think the only way is to develop a compiled FILE module function that wrap CopyFileEx allowing passing a script function that will be called when needed.

Will see what I can do but not just round the corner.

Would a freeBasic #COMPILED section do the work ? I haven't listed if it's about CopyFileEx only.

Edit2: https://www.freebasic.net/wiki/wikka...KeyPgOpProcptr

[DirectuX]

At least , I can think of 'move' function that can be long too. If I'm not mistaken , others operations (rename, delete...) can be managed (synchronously = script blocking) with standard thinBasic functions.

[ErosOlmi]

I've made some tests and maybe I will have something for tomorrow.

[DirectuX]

how quick !

[ErosOlmi]

Updated thinBasic 1.11.3 at https://www.thinbasic.biz/projects/t...c_1.11.3.0.zip

Added FILE_CopyEX function (not yet documented) that partially wrap CopyFileExA https://docs.microsoft.com/en-us/win...se-copyfileexa

How to use?
Check example in \thinBasic\SampleScripts\File\File_CopyEX.tbasic reported here below.
Copy one file into destination and automatically call a "CopyProgressRoutine" callback function during copy execution in order to have the option to progress some info or cancel operation.
IMPORTANT "CopyProgressRoutine" callback function can have any name but MUST EXACTLY be defined like in the example (otherwise script will GPF)

I've tested with files larger up to 140MB as seems very fast.
Data chunk size cannot be controlled.

If I'm on the right track I will wrap CopyFileExW and possibly add more options.
Let me know if it what you needed.

Ciao
Eros

'---References
'     https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-copyfileexa
'     https://docs.microsoft.com/it-it/windows/win32/api/winbase/nc-winbase-lpprogress_routine
'---------------------------------------------------------------------------------------------


uses "Console"
uses "File"


  callback Function CopyProgressRoutine( 
                                  ByVal TotalFileSize           As Quad,   ' // total file size, in bytes
                                  ByVal TotalBytesTransferred   As Quad,   ' // total number of bytes transferred
                                  ByVal StreamSize              As Quad,   ' // total number of bytes for this stream
                                  ByVal StreamBytesTransferred  As Quad,   ' // total number of bytes transferred for this stream
                                  ByVal dwStreamNumber          As Long,   ' // the current stream
                                  ByVal dwCallbackReason        As Long,   ' // reason for callback
                                  ByVal hSourceFile             As Long,   ' // handle to the source file
                                  ByVal hDestinationFile        As Long,   ' // handle to the destination file
                                  ByVal lpData                  As Long  ) As Long


  'Function should send back one of the following:
  ' %FILE_PROGRESS_CONTINUE  Continue the copy operation.
  ' %FILE_PROGRESS_CANCEL    Cancel the copy operation and delete the destination file.
  ' %FILE_PROGRESS_STOP      Stop the copy operation. It can be restarted at a later time.
  ' %FILE_PROGRESS_QUIET     Continue the copy operation, but stop invoking CopyProgressRoutine to report progress.


    printl  TotalFileSize, TotalBytesTransferred At 10, 10
    
    function = %FILE_PROGRESS_CONTINUE


  end Function


string sFileSource      = APP_ScriptFullName
string sFileDestination = sFileSource & ".File_CopyEX.txt"
long   lRet


printl "Copying: " & sFileSource
printl "To.....: " & sFileDestination


lRet = file_copyex(sFileSource, sFileDestination, CopyProgressRoutine)


printl "Result.: " & lret


printl "All done. Press a key to end"
WaitKey