Page 2 of 2 FirstFirst 12
Results 11 to 20 of 20

Thread: asynchronous file operations

  1. #11
    Member DirectuX's Avatar
    Join Date
    Oct 2018
    Location
    France
    Posts
    417
    Rep Power
    55
    Quote Originally Posted by ErosOlmi View Post
    Updated thinBasic 1.11.3 at https://www.thinbasic.biz/projects/t...c_1.11.3.0.zip

    Added FILE_CopyEX function (not yet documented) that partially wrap CopyFileExA https://docs.microsoft.com/en-us/win...se-copyfileexa

    How to use?
    Check example in \thinBasic\SampleScripts\File\File_CopyEX.tbasic reported here below.
    Copy one file into destination and automatically call a "CopyProgressRoutine" callback function during copy execution in order to have the option to progress some info or cancel operation.
    IMPORTANT "CopyProgressRoutine" callback function can have any name but MUST EXACTLY be defined like in the example (otherwise script will GPF)

    I've tested with files larger up to 140MB as seems very fast.
    I tested it. Yes, fast for regular files.
    I tested it for GB files too. Works.

    From documentation
    dwCopyFlags
    Flags that specify how the file is to be copied.
    Value Meaning
    COPY_FILE_NO_BUFFERING
    0x00001000
    The copy operation is performed using unbuffered I/O, bypassing system I/O cache resources. Recommended for very large file transfers.


    Quote Originally Posted by ErosOlmi View Post
    Data chunk size cannot be controlled.
    That would be for optimization against sector size. (Former I've seen noticeable speed improvement when adjusting this for RAID disks; today I've no RAID)

    Quote Originally Posted by ErosOlmi View Post
    If I'm on the right track I will wrap CopyFileExW and possibly add more options.
    Let me know if it what you needed.
    Ah ! I've missed something. I thought that with callbacks coding style, script would be able to continue while the filesystem operations are processed, and be notified regularly on process progress. In fact, the script waits for the copy to end.

    Quote Originally Posted by DirectuX View Post
    thinBasic\SampleScripts\UI\Power\Power_Messages.tBasic is a sample that can detect power status changes via callback management.
    I'm in search of such behaviour for complete filesystem interaction.
    As a start I studied the file copy feature:
    Both FILE_Copy and FILE_ShellCopy are synchronous functions. None can expose the copy progression to the script.
    So... having a progression feedback, we are at half way.
    Quote Originally Posted by DirectuX View Post
    Before moving in any direction, I would like to ask (...)
    I think I mislead myself with ideas like DIALOG SHOW MODELESS instruction and callbacks. That is one reason I started questioning
    Now, I foresee even more the difficulty to have asynchronous file operation.

    Quote Originally Posted by ErosOlmi View Post
    Let me know if it what you needed.
    Except the non script-blocking disk-operations that is essential.




    For information. My project is not alpha ready but on right track.
    Roughly : I have:
    1 filesystem(FS) unit
    1 DB unit
    1 UI unit (no started yet)

    user can work offline to queue operations (disks and shares can be offline), by interacting on DB through UI.
    script tries to execute FS operations as soon as possible ( rename, copie, move, delete) while not blocking user work
    script maintain db up to date. (not yet explored)

    thinking...

    maybe I should split the project in two scripts ? (I'd rather not)

    I want to start simple (manual db update) and add feature on progress.

    Features for alpha :
    manual db update
    display (2 panes) disks folder tree + files + duplicates
    basic file operations




    PS : not going into multi-threading like xLeaves's LzBot , is this concept possible in thinBasic ? (this is single thread)
    Last edited by DirectuX; 06-02-2020 at 18:17.
    ThinBasic 1.11.6.0 ALPHA - Windows 8.1 x64

  2. #12
    thinBasic author ErosOlmi's Avatar
    Join Date
    Sep 2004
    Location
    Milan - Italy
    Age
    57
    Posts
    8,818
    Rep Power
    10
    Maybe you need a multy thread programming language so you can run one thread for each file copy operation and have one main thread controlling the process of child threads.

    At the moment thinBasic is not able to execute part of the script (for example a function) in a separate thread.
    One way could be to use FreeBasic compiled code, have thinBasic script as orchestrator and FreeBasic run copy threads.

    Script \thinBasic\SampleScripts\FreeBASIC\FB_Sample_04_PrintAndThread.tBasic can be a start to look at.
    Main thinBasic script waits and show results while compiled FreeBasic code execute a thread changing a value.
    Last edited by ErosOlmi; 06-02-2020 at 20:16.
    www.thinbasic.com | www.thinbasic.com/community/ | help.thinbasic.com
    Windows 10 Pro for Workstations 64bit - 32 GB - Intel(R) Xeon(R) W-10855M CPU @ 2.80GHz - NVIDIA Quadro RTX 3000

  3. #13
    Member DirectuX's Avatar
    Join Date
    Oct 2018
    Location
    France
    Posts
    417
    Rep Power
    55
    Thanks for your guidance Eros,

    I'll keep you informed.
    ThinBasic 1.11.6.0 ALPHA - Windows 8.1 x64

  4. #14
    Member DirectuX's Avatar
    Join Date
    Oct 2018
    Location
    France
    Posts
    417
    Rep Power
    55
    Quote Originally Posted by ErosOlmi View Post
    I think the API to use is CopyFileExA
    https://docs.microsoft.com/en-us/win...se-copyfileexa
    My mistake, it is nowhere written that CopyFileEx returns immediately. It is just said that it has progress report.
    Still studying... win32/fileio/synchronous-and-asynchronous-i-o,
    win32/ipc/synchronous-and-overlapped-input-and-output

    Quote Originally Posted by ErosOlmi View Post
    Maybe you need a multy thread programming language
    The more I read, the more I understand that it's not the sole solution. -> multithreading -> multitasking

    Quote Originally Posted by ErosOlmi View Post
    One way could be to use FreeBasic compiled code, have thinBasic script as orchestrator and FreeBasic run copy threads.
    Script \thinBasic\SampleScripts\FreeBASIC\FB_Sample_04_PrintAndThread.tBasic can be a start to look at.
    Main thinBasic script waits and show results while compiled FreeBasic code execute a thread changing a value.
    Looked at. Share your thought... for last resort.
    ThinBasic 1.11.6.0 ALPHA - Windows 8.1 x64

  5. #15
    Member DirectuX's Avatar
    Join Date
    Oct 2018
    Location
    France
    Posts
    417
    Rep Power
    55
    As a start.

    I'm unsure if the declarations in this draft are correct.
    The buffer seems empty.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    133
    134
    135
    136
    137
    138
    139
    140
    141
    142
    uses "Console"
    uses "File"
     
    #Region "Declarations"
     
       
      %INVALID_HANDLE_VALUE = &HFFFFFFFF as dword
     
      %GENERIC_READ = 0x80000000 as dword ' Specifies access control suitable for reading the object.
      %GENERIC_WRITE = 0x40000000 as dword ' Specifies access control suitable for updating attributes on the object.
     
      %ERROR_FILE_NOT_FOUND = 2
      %FILE_SHARE_DELETE = 0x00000004 as dword ' Enables subsequent open operations on a file or device to request delete access. Otherwise, other processes cannot open the file or device if they request delete access.
      %FILE_SHARE_READ = 0x00000001 as dword ' Enables subsequent open operations on a file or device to request read access. Otherwise, other processes cannot open the file or device if they request read access.
      %FILE_SHARE_WRITE = 0x00000002 as dword ' Enables subsequent open operations on a file or device to request write access. Otherwise, other processes cannot open the file or device if they request write access.
      %OPEN_EXISTING = 3 as dword ' Opens a file or device, only if it exists.
      %FILE_FLAG_SEQUENTIAL_SCAN = 0x08000000  as dword ' Access is intended to be sequential from beginning to end.
      %FILE_ATTRIBUTE_SYSTEM = 4 as dword ' The file is part of or used exclusively by an operating system.
      %FILE_FLAG_OVERLAPPED = 0x40000000 as DWord ' The file or device is being opened or created for asynchronous I/O. When subsequent I/O operations are completed on this handle, the event specified in the OVERLAPPED structure will be set to the signaled state.
      %FILE_FLAG_NO_BUFFERING = 0x20000000 as DWord 'The file or device is being opened with no system caching for data reads and writes. This flag does not affect hard disk caching or memory mapped files.
     
      Type overlapped
        Internal as quad ' The status code for the I/O request. When the request is issued, the system sets this member to STATUS_PENDING to indicate that the operation has not yet started. When the request is completed, the system sets this member to the status code for the completed request.
        InternalHigh as quad ' The number of bytes transferred for the I/O request. The system sets this member if the request is completed without errors.
        Offset as DWord ' The low-order portion of the file position at which to start the I/O request, as specified by the user. This member is nonzero only when performing I/O requests on a seeking device that supports the concept of an offset (also referred to as a file pointer mechanism), such as a file. Otherwise, this member must be zero.
        OffsetHigh as DWord ' The high-order portion of the file position at which to start the I/O request, as specified by the user. This member is nonzero only when performing I/O requests on a seeking device that supports the concept of an offset (also referred to as a file pointer mechanism), such as a file. Otherwise, this member must be zero.
        Pointer as Long' Reserved for system use; do not use after initialization to zero.
        hEvent as long' A handle to the event that will be set to a signaled state by the system when the operation has completed.
      end type
       
       
      TYPE SECURITY_ATTRIBUTES DWORD
        nLength AS DWORD
        lpSecurityDescriptor AS DWORD
        bInheritHandle AS LONG
      END TYPE
     
      Declare Function ReadFileEx lib "KERNEL32.DLL" Alias "ReadFileEx" (
                                                                        ByVal hFile As Long,                  ' A handle to the file or I/O device
                                                                        byref lpBuffer As Asciiz ,            ' A pointer to a buffer that receives the data read from the file or device.
                                                                        ByVal nNumberOfBytesToRead As Long,   ' The number of bytes to be read.
                                                                        byref lpOverlapped As overlapped,     ' A pointer to an OVERLAPPED data structure that supplies data to be used during the asynchronous (overlapped) file read operation.
                                                                        ByVal lpCompletionRoutine As long     ' A pointer to the completion routine to be called when the read operation is complete and the calling thread is in an alertable wait state.
                                                                        ) As Long
     
      DECLARE FUNCTION CreateFileA lib "KERNEL32.DLL" ALIAS "CreateFileA" (
                                                                          byref lpFileName AS ASCIIZ,                         ' The name of the file or device to be created or opened.
                                                                          byval dwDesiredAccess AS DWORD,                     ' The requested access to the file or device, which can be summarized as read, write, both or neither zero).
                                                                          byval dwShareMode AS DWORD,                         ' The requested sharing mode of the file or device, which can be read, write, both, delete, all of these, or none
                                                                          byref lpSecurityAttributes AS SECURITY_ATTRIBUTES,  ' A pointer to a SECURITY_ATTRIBUTES structure. This parameter can be NULL.
                                                                          byval dwCreationDisposition AS DWORD,               ' An action to take on a file or device that exists or does not exist.
                                                                          byval dwFlagsAndAttributes AS DWORD,                ' The file or device attributes and flags, FILE_ATTRIBUTE_NORMAL being the most common default value for files.
                                                                          byval hTemplateFile AS DWORD                        ' A valid handle to a template file with the GENERIC_READ access right. This parameter can be NULL.
                                                                          ) AS DWORD                                          ' If the function succeeds, the return value is an open handle to the specified file, device, named pipe, or mail slot.
     
      Declare Function CloseHandle lib "KERNEL32.DLL" Alias "CloseHandle" (byval hObject as DWORD) AS LONG
     
    #EndRegion
     
    function LpoverlappedCompletionRoutine(
                                          byval dwErrorCode as DWord,               ' The I/O completion status. This parameter can be one of the system error codes.
                                          Byval dwNumberOfBytesTransfered as DWORD' The number of bytes transferred. If an error occurs, this parameter is zero.
                                          byval lpOverlapped as long                 ' A pointer to the OVERLAPPED structure specified by the asynchronous I/O function.
                                          ) as Long
      Printl lpOverlapped.InternalHigh
      printl dwErrorCode
      printl dwNumberOfBytesTransfered
     
      return 0
    end Function
     
     
    '
    '  DIALOG SHOW MODELESS hDlg Call DlgCallback
    '
    '  Do
    '
    '    DIALOG DOEVENTS 0 To Count
    '
    '    if not EOF
    '        if not cancelled
    '           doReadNextBlock
    '
    '  Loop While Count
    '
     
     
    Function TBMain() as Long
     
      Local sFileSource as asciiz ' Full path to file to be read
      local voverlapped as overlapped
      Local hFile as Long ' handle to file to be read
      Local lpSecurityAttributes as SECURITY_ATTRIBUTES
      Local success as long
      Local lpBuffer as asciiz * 512 ' cluster size
      Local result as long
       
      Global fileSize as DWord
     
      sFileSource = "t:\test.txt"       ' File to read
      fileSize = FILE_Size(sFileSource)
       
      Printl "File : " & sFileSource
      Printl "Size (Bytes) : " & fileSize
     
      hFile = CreateFileA(sFileSource, %GENERIC_READ, %FILE_SHARE_READ, lpSecurityAttributes, %OPEN_EXISTING, %FILE_FLAG_NO_BUFFERING or %FILE_FLAG_OVERLAPPED, Null)
       
      if hFile <> %INVALID_HANDLE_VALUE then
       
        Printl "File handle = " & hFile
         
        result = ReadFileEx(hFile, lpBuffer, 512, voverlapped, CodePtr(LpoverlappedCompletionRoutine))
         
        printl "ReadFileEx result (<> 0 means no error) : " & result
        Printl "Buffer dump : " & lpBuffer
     
      Else
        Printl "Failed to get a file handle." in %CONSOLE_FOREGROUND_RED or %CONSOLE_FOREGROUND_INTENSITY
        success = %FALSE
      endif
     
      closeHandle(hFile)
       
      printl "Press a key to end."
       
    WaitKey
     
      if success = %FALSE then
        Return %FALSE
      Else
        Return %TRUE
      endif
     
    end function
    ThinBasic 1.11.6.0 ALPHA - Windows 8.1 x64

  6. #16
    Member DirectuX's Avatar
    Join Date
    Oct 2018
    Location
    France
    Posts
    417
    Rep Power
    55
    Progress...

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    133
    134
    135
    136
    137
    138
    139
    140
    141
    142
    143
    144
    145
    146
    147
    148
    149
    150
    151
    152
    153
    uses "Console"
    uses "File"
     
    #Region "Declarations"
     
       
      %INVALID_HANDLE_VALUE = &HFFFFFFFF as dword
     
      %GENERIC_READ = 0x80000000 as dword ' Specifies access control suitable for reading the object.
      %GENERIC_WRITE = 0x40000000 as dword ' Specifies access control suitable for updating attributes on the object.
     
      %ERROR_FILE_NOT_FOUND = 2
      %FILE_SHARE_DELETE = 0x00000004 as dword ' Enables subsequent open operations on a file or device to request delete access. Otherwise, other processes cannot open the file or device if they request delete access.
      %FILE_SHARE_READ = 0x00000001 as dword ' Enables subsequent open operations on a file or device to request read access. Otherwise, other processes cannot open the file or device if they request read access.
      %FILE_SHARE_WRITE = 0x00000002 as dword ' Enables subsequent open operations on a file or device to request write access. Otherwise, other processes cannot open the file or device if they request write access.
      %OPEN_EXISTING = 3 as dword ' Opens a file or device, only if it exists.
      %FILE_FLAG_SEQUENTIAL_SCAN = 0x08000000  as dword ' Access is intended to be sequential from beginning to end.
      %FILE_ATTRIBUTE_SYSTEM = 4 as dword ' The file is part of or used exclusively by an operating system.
      %FILE_FLAG_OVERLAPPED = 0x40000000 as DWord ' The file or device is being opened or created for asynchronous I/O. When subsequent I/O operations are completed on this handle, the event specified in the OVERLAPPED structure will be set to the signaled state.
      %FILE_FLAG_NO_BUFFERING = 0x20000000 as DWord 'The file or device is being opened with no system caching for data reads and writes. This flag does not affect hard disk caching or memory mapped files.
     
      Type overlapped
        Internal as quad ' The status code for the I/O request. When the request is issued, the system sets this member to STATUS_PENDING to indicate that the operation has not yet started. When the request is completed, the system sets this member to the status code for the completed request.
        InternalHigh as quad ' The number of bytes transferred for the I/O request. The system sets this member if the request is completed without errors.
        Offset as DWord ' The low-order portion of the file position at which to start the I/O request, as specified by the user. This member is nonzero only when performing I/O requests on a seeking device that supports the concept of an offset (also referred to as a file pointer mechanism), such as a file. Otherwise, this member must be zero.
        OffsetHigh as DWord ' The high-order portion of the file position at which to start the I/O request, as specified by the user. This member is nonzero only when performing I/O requests on a seeking device that supports the concept of an offset (also referred to as a file pointer mechanism), such as a file. Otherwise, this member must be zero.
        Pointer as Long' Reserved for system use; do not use after initialization to zero.
        hEvent as long' A handle to the event that will be set to a signaled state by the system when the operation has completed.
      end type
       
       
      TYPE SECURITY_ATTRIBUTES DWORD
        nLength AS DWORD
        lpSecurityDescriptor AS DWORD
        bInheritHandle AS LONG
      END TYPE
     
      Declare Function ReadFileEx lib "KERNEL32.DLL" Alias "ReadFileEx" (
                                                                        ByVal hFile As Long,                  ' A handle to the file or I/O device
                                                                        byref lpBuffer As Asciiz ,            ' A pointer to a buffer that receives the data read from the file or device.
                                                                        ByVal nNumberOfBytesToRead As Long,   ' The number of bytes to be read.
                                                                        byref lpOverlapped As overlapped,     ' A pointer to an OVERLAPPED data structure that supplies data to be used during the asynchronous (overlapped) file read operation.
                                                                        ByVal lpCompletionRoutine As long     ' A pointer to the completion routine to be called when the read operation is complete and the calling thread is in an alertable wait state.
                                                                        ) As Long
     
      DECLARE FUNCTION CreateFileA lib "KERNEL32.DLL" ALIAS "CreateFileA" (
                                                                          byref lpFileName AS ASCIIZ,                         ' The name of the file or device to be created or opened.
                                                                          byval dwDesiredAccess AS DWORD,                     ' The requested access to the file or device, which can be summarized as read, write, both or neither zero).
                                                                          byval dwShareMode AS DWORD,                         ' The requested sharing mode of the file or device, which can be read, write, both, delete, all of these, or none
                                                                          byref lpSecurityAttributes AS SECURITY_ATTRIBUTES,  ' A pointer to a SECURITY_ATTRIBUTES structure. This parameter can be NULL.
                                                                          byval dwCreationDisposition AS DWORD,               ' An action to take on a file or device that exists or does not exist.
                                                                          byval dwFlagsAndAttributes AS DWORD,                ' The file or device attributes and flags, FILE_ATTRIBUTE_NORMAL being the most common default value for files.
                                                                          byval hTemplateFile AS DWORD                        ' A valid handle to a template file with the GENERIC_READ access right. This parameter can be NULL.
                                                                          ) AS DWORD                                          ' If the function succeeds, the return value is an open handle to the specified file, device, named pipe, or mail slot.
     
      Declare Function CloseHandle lib "KERNEL32.DLL" Alias "CloseHandle" (byval hObject as DWORD) AS LONG
     
      declare function SleepEx lib "KERNEL32.DLL" Alias "SleepEx" (
                                                                  byval dwMilliseconds as dword,
                                                                  byval bAlertable as long
                                                                  )as Long
      #EndRegion
     
    function LpoverlappedCompletionRoutine(
                                          byval dwErrorCode as DWord,               ' The I/O completion status. This parameter can be one of the system error codes.
                                          Byval dwNumberOfBytesTransfered as DWORD' The number of bytes transferred. If an error occurs, this parameter is zero.
                                          byval lpOverlapped as long                 ' A pointer to the OVERLAPPED structure specified by the asynchronous I/O function.
                                          ) as Long
       
      printl "inside LpoverlappedCompletionRoutine"
     
      printl "errCode = " & dwErrorCode
      printl "Bytes read = " & dwNumberOfBytesTransfered
     
      return 0
    end Function
     
     
    '
    '  DIALOG SHOW MODELESS hDlg Call DlgCallback
    '
    '  Do
    '
    '    DIALOG DOEVENTS 0 To Count
    '
    '    if not EOF
    '        if not cancelled
    '           doReadNextBlock
    '
    '  Loop While Count
    '
     
     
    Function TBMain() as Long
     
      Global sFileSource as asciiz ' Full path to file to be read
      Global voverlapped as overlapped
      Global hFile as Long ' handle to file to be read
      Global lpSecurityAttributes as SECURITY_ATTRIBUTES
      Global success as long
      Global lpBuffer as asciiz * 512 ' cluster size
      Global result as long
       
      Global fileSize as DWord
     
      sFileSource = "t:\test.txt"       ' File to read
      fileSize = FILE_Size(sFileSource)
       
      Printl "File : " & sFileSource
      Printl "Size (Bytes) : " & fileSize
     
      hFile = CreateFileA(sFileSource, %GENERIC_READ, %FILE_SHARE_READ, lpSecurityAttributes, %OPEN_EXISTING, %FILE_FLAG_NO_BUFFERING or %FILE_FLAG_OVERLAPPED, Null)
       
      if hFile <> %INVALID_HANDLE_VALUE then
       
        Printl "File handle = " & hFile
         
        result = ReadFileEx(hFile, lpBuffer, 512, voverlapped, CodePtr(LpoverlappedCompletionRoutine))
         
        printl "ReadFileEx result (<> 0 means no error) : " & result
         
        SleepEx(100,%TRUE)
        Printl "Buffer dump : " & lpBuffer
     
      Else
        Printl "Failed to get a file handle." in %CONSOLE_FOREGROUND_RED or %CONSOLE_FOREGROUND_INTENSITY
        success = %FALSE
      endif
     
       
       
      printl "Press a key to end."
      DoEvents
    WaitKey
     
    closeHandle(hFile)
     
      if success = %FALSE then
        Return %FALSE
      Else
        Return %TRUE
      endif
     
    end function
    ThinBasic 1.11.6.0 ALPHA - Windows 8.1 x64

  7. #17
    Member DirectuX's Avatar
    Join Date
    Oct 2018
    Location
    France
    Posts
    417
    Rep Power
    55
    More progress !

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    133
    134
    135
    136
    137
    138
    139
    140
    141
    142
    143
    144
    145
    146
    147
    148
    149
    150
    151
    152
    153
    154
    155
    156
    157
    158
    159
    160
    161
    162
    163
    164
    165
    166
    167
    168
    169
    170
    171
    172
    173
    174
    175
    176
    177
    178
    179
    180
    181
    182
    183
    184
    185
    186
    187
    188
    189
    190
    191
    192
    193
    194
    195
    196
    197
    198
    199
    200
    201
    202
    203
    204
    205
    uses "Console"
    uses "File"
    uses "UI"
     
    dim myTime as DWord = Timer
     
    #Region "Declarations"
       
      %INVALID_HANDLE_VALUE = &HFFFFFFFF as dword
     
      %GENERIC_READ = 0x80000000 as dword ' Specifies access control suitable for reading the object.
      %GENERIC_WRITE = 0x40000000 as dword ' Specifies access control suitable for updating attributes on the object.
     
      %ERROR_FILE_NOT_FOUND = 2
      %FILE_SHARE_DELETE = 0x00000004 as dword ' Enables subsequent open operations on a file or device to request delete access. Otherwise, other processes cannot open the file or device if they request delete access.
      %FILE_SHARE_READ = 0x00000001 as dword ' Enables subsequent open operations on a file or device to request read access. Otherwise, other processes cannot open the file or device if they request read access.
      %FILE_SHARE_WRITE = 0x00000002 as dword ' Enables subsequent open operations on a file or device to request write access. Otherwise, other processes cannot open the file or device if they request write access.
      %OPEN_EXISTING = 3 as dword ' Opens a file or device, only if it exists.
      %FILE_FLAG_SEQUENTIAL_SCAN = 0x08000000  as dword ' Access is intended to be sequential from beginning to end.
      %FILE_ATTRIBUTE_SYSTEM = 4 as dword ' The file is part of or used exclusively by an operating system.
      %FILE_FLAG_OVERLAPPED = 0x40000000 as DWord ' The file or device is being opened or created for asynchronous I/O. When subsequent I/O operations are completed on this handle, the event specified in the OVERLAPPED structure will be set to the signaled state.
      %FILE_FLAG_NO_BUFFERING = 0x20000000 as DWord 'The file or device is being opened with no system caching for data reads and writes. This flag does not affect hard disk caching or memory mapped files.
     
      Type overlapped
        Internal as quad ' The status code for the I/O request. When the request is issued, the system sets this member to STATUS_PENDING to indicate that the operation has not yet started. When the request is completed, the system sets this member to the status code for the completed request.
        InternalHigh as quad ' The number of bytes transferred for the I/O request. The system sets this member if the request is completed without errors.
        Offset as DWord ' The low-order portion of the file position at which to start the I/O request, as specified by the user. This member is nonzero only when performing I/O requests on a seeking device that supports the concept of an offset (also referred to as a file pointer mechanism), such as a file. Otherwise, this member must be zero.
        OffsetHigh as DWord ' The high-order portion of the file position at which to start the I/O request, as specified by the user. This member is nonzero only when performing I/O requests on a seeking device that supports the concept of an offset (also referred to as a file pointer mechanism), such as a file. Otherwise, this member must be zero.
        Pointer as Long' Reserved for system use; do not use after initialization to zero.
        hEvent as long' A handle to the event that will be set to a signaled state by the system when the operation has completed.
      end type
       
      TYPE SECURITY_ATTRIBUTES DWORD
        nLength AS DWORD
        lpSecurityDescriptor AS DWORD
        bInheritHandle AS LONG
      END TYPE
     
      Declare Function ReadFileEx lib "KERNEL32.DLL" Alias "ReadFileEx" (
                                                                        ByVal hFile As Long,                  ' A handle to the file or I/O device
                                                                        byref lpBuffer As string ,            ' A pointer to a buffer that receives the data read from the file or device.
                                                                        ByVal nNumberOfBytesToRead As Long,   ' The number of bytes to be read.
                                                                        byref lpOverlapped As overlapped,     ' A pointer to an OVERLAPPED data structure that supplies data to be used during the asynchronous (overlapped) file read operation.
                                                                        ByVal lpCompletionRoutine As long     ' A pointer to the completion routine to be called when the read operation is complete and the calling thread is in an alertable wait state.
                                                                        ) As Long
     
      DECLARE FUNCTION CreateFileA lib "KERNEL32.DLL" ALIAS "CreateFileA" (
                                                                          byref lpFileName AS ASCIIZ,                         ' The name of the file or device to be created or opened.
                                                                          byval dwDesiredAccess AS DWORD,                     ' The requested access to the file or device, which can be summarized as read, write, both or neither zero).
                                                                          byval dwShareMode AS DWORD,                         ' The requested sharing mode of the file or device, which can be read, write, both, delete, all of these, or none
                                                                          byref lpSecurityAttributes AS SECURITY_ATTRIBUTES,  ' A pointer to a SECURITY_ATTRIBUTES structure. This parameter can be NULL.
                                                                          byval dwCreationDisposition AS DWORD,               ' An action to take on a file or device that exists or does not exist.
                                                                          byval dwFlagsAndAttributes AS DWORD,                ' The file or device attributes and flags, FILE_ATTRIBUTE_NORMAL being the most common default value for files.
                                                                          byval hTemplateFile AS DWORD                        ' A valid handle to a template file with the GENERIC_READ access right. This parameter can be NULL.
                                                                          ) AS DWORD                                          ' If the function succeeds, the return value is an open handle to the specified file, device, named pipe, or mail slot.
     
      Declare Function CloseHandle lib "KERNEL32.DLL" Alias "CloseHandle" (byval hObject as DWORD) AS LONG
     
      declare function SleepEx lib "KERNEL32.DLL" Alias "SleepEx" (
                                                                  byval dwMilliseconds as dword,
                                                                  byval bAlertable as long
                                                                  )as Long
      #EndRegion
     
    function LpoverlappedCompletionRoutine(
                                          byval dwErrorCode as DWord,               ' The I/O completion status. This parameter can be one of the system error codes.
                                          Byval dwNumberOfBytesTransfered as DWORD' The number of bytes transferred. If an error occurs, this parameter is zero.
                                          byval lpOverlapped as long                 ' A pointer to the OVERLAPPED structure specified by the asynchronous I/O function.
                                          ) as Long
       
      printl "inside LpoverlappedCompletionRoutine"
     
      printl "errCode = " & dwErrorCode
      printl "Bytes read = " & dwNumberOfBytesTransfered
       
      CONTROL SET TEXT hDlg, %label2, lpBuffer
     
      return 0
    end Function
     
    Begin ControlID
      %Label1
      %Label2
      %Button1
    End ControlID
     
     
    '
    '  DIALOG SHOW MODELESS hDlg Call DlgCallback
    '
    '  Do
    '
    '    DIALOG DOEVENTS 0 To Count
    '
    '    if not EOF
    '        if not cancelled
    '           doReadNextBlock
    '
    '  Loop While Count
    '
    CALLBACK Function DlgCallback() As Long
      Select Case CBMSG
        Case %WM_Command
          If CBCTLMSG = %BN_CLICKED Then
            SELECT CASE cbctl
              case %Button1
                call start_read
            end Select
          endif
        Case %WM_DESTROY
          MSGBOX 0, "Window is to be destroyed."
      End Select
    End Function
     
    function start_read()
     
      Global voverlapped as overlapped
      Global hFile as Long ' handle to file to be read
      Global lpBuffer as string * 512 ' cluster size
       
      Local lpSecurityAttributes as SECURITY_ATTRIBUTES
      Local result as long
      Local success as long
     
      hFile = CreateFileA(sFileSource, %GENERIC_READ, %FILE_SHARE_READ, lpSecurityAttributes, %OPEN_EXISTING, %FILE_FLAG_NO_BUFFERING or %FILE_FLAG_OVERLAPPED, Null)
       
      if hFile <> %INVALID_HANDLE_VALUE then
       
        Printl "File handle = " & hFile
        result = ReadFileEx(hFile, lpBuffer, 512, voverlapped, CodePtr(LpoverlappedCompletionRoutine))
        readInProgress = %TRUE
        printl "ReadFileEx result (<> 0 means no error) : " & result
     
      Else
        Printl "Failed to get a file handle." in %CONSOLE_FOREGROUND_RED or %CONSOLE_FOREGROUND_INTENSITY
        success = %FALSE
      endif
       
      if success = %FALSE then
        Return %FALSE
      Else
        Return %TRUE
      endif
       
     
    end Function
     
    Function TBMain() as Long
     
      Local count as Integer
      Global hDlg as DWord ' Dialog holder
       
      Global sFileSource as asciiz ' Full path to file to be read
      Global fileSize as DWord
      Global readInProgress as Boolean
       
      sFileSource = "t:\pg200.txt"       ' File to read
      fileSize = FILE_Size(sFileSource)
       
      Printl "File : " & sFileSource
      Printl "Size (Bytes) : " & fileSize
       
      call MakeUi
     
      Do
        DIALOG DOEVENTS 0 To Count
        SleepEx(1,%TRUE)
        updateLabel1
      Loop While Count
       
      closeHandle(hFile)
     
    end function
     
    Function updateLabel1()
     
      Local sLabel1 as String
      Local nTime as DWord
       
      nTime = Timer - myTime
      sLabel1 = "Script is running since : " & nTime & " seconds"
      CONTROL SET TEXT hDlg, %label1, sLabel1
     
    end function
     
    Function MakeUi()
     
      DIALOG NEW 0, "form1", -1, -1, 195, 180, %WS_DLGFRAME OR %DS_CENTER OR %WS_CAPTION OR %WS_SYSMENU, 0 TO hDlg
     
      CONTROL ADD Label , hDlg, %label1, "...",   5,  15, 185, 15, %WS_BORDER OR %ES_RIGHT, %WS_EX_CLIENTEDGE
      CONTROL ADD Label , hDlg, %label2, "...",   5,  35, 185, 15, %WS_BORDER OR %ES_RIGHT, %WS_EX_CLIENTEDGE
      CONTROL ADD BUTTON  , hDlg, %button1,"Start READ",  65,  80,  45, 20
       
      DIALOG SHOW MODELESS hDlg Call DlgCallback
     
    end function
    Have still to find how to prog read_next_chunk function.

    Note: have to explore (show dialog modal + dialog set timer) as an alternative

    Eros, side question : how to show dialog modeless with #resource ? , I didn't find.
    Last edited by DirectuX; 09-02-2020 at 11:51.
    ThinBasic 1.11.6.0 ALPHA - Windows 8.1 x64

  8. #18
    Member DirectuX's Avatar
    Join Date
    Oct 2018
    Location
    France
    Posts
    417
    Rep Power
    55
    Finally,

    this sample shows that thinbasic is capable of doing non-blocking file operation.
    but : in this state this is slow,
    I can't increase the chunk size, I don't know why (20480 bytes is maximum).
    Pipe is not implemented in this example.

    Again, I ask opinion from TB community.
    Anyone interested in this challenge ?

    Archive.zip
    ThinBasic 1.11.6.0 ALPHA - Windows 8.1 x64

  9. #19
    Member DirectuX's Avatar
    Join Date
    Oct 2018
    Location
    France
    Posts
    417
    Rep Power
    55

    Thumbs up

    Quote Originally Posted by ErosOlmi View Post
    Updated thinBasic 1.11.3 at https://www.thinbasic.biz/projects/t...c_1.11.3.0.zip

    Added FILE_CopyEX function (not yet documented) that partially wrap CopyFileExA https://docs.microsoft.com/en-us/win...se-copyfileexa

    How to use?
    Check example in \thinBasic\SampleScripts\File\File_CopyEX.tbasic reported here below.
    Copy one file into destination and automatically call a "CopyProgressRoutine" callback function during copy execution in order to have the option to progress some info or cancel operation.
    IMPORTANT "CopyProgressRoutine" callback function can have any name but MUST EXACTLY be defined like in the example (otherwise script will GPF)

    I've tested with files larger up to 140MB as seems very fast.
    Data chunk size cannot be controlled.

    If I'm on the right track I will wrap CopyFileExW and possibly add more options.
    Let me know if it what you needed.

    Ciao
    Eros


    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    '---References
    '---------------------------------------------------------------------------------------------
     
     
    uses "Console"
    uses "File"
     
     
      callback Function CopyProgressRoutine( ...

    I made a sample based upon this script and this thinBasic very version. One can have a GUI and at the same time make non blocking copies and follow progress.

    async.zip

    Note about the pause feature: Eros, if possible, can we have a discussion on the Pause feature ?
    1
    '%FILE_PROGRESS_STOP      Stop the copy operation. It can be restarted at a later time.
    It is not working as I expected, because if %FILE_PROGRESS_STOP is returned, the file_copyex ends.
    How do you think it is intended to implement the copy continue ?
    ThinBasic 1.11.6.0 ALPHA - Windows 8.1 x64

  10. #20
    Member DirectuX's Avatar
    Join Date
    Oct 2018
    Location
    France
    Posts
    417
    Rep Power
    55
    Quote Originally Posted by ErosOlmi View Post
    No, didn't change anything in that area.
    I'm working mostly in thinAir

    Can you send me a complete code example giving error so I can test it?
    Eros, you can use the complete script in this above #19 post (it's the modal version but it has the same thindebug error as the modeless one)

    Edit: here is the modeless version Nouveau dossier.zip, but I think I will continue dev with the modal version.
    Last edited by DirectuX; 28-03-2020 at 19:45.
    ThinBasic 1.11.6.0 ALPHA - Windows 8.1 x64

Page 2 of 2 FirstFirst 12

Similar Threads

  1. Demonstration of matrix operations
    By Petr Schreiber in forum TBGL Tutorials
    Replies: 4
    Last Post: 30-03-2009, 16:24

Members who have read this thread: 1

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •