Delphi Service Application crashing after several hours (wodSFTP / wodSFTP.NET / wodSFTPdll)

Hi,
I'm using wodSFTP (imported as a Type Library) in a Win32 Service Application developed in Delphi XE2.

The Application connects to an SFTP Server every 15 minutes and checks for new files. If there are any, it downloads them and then deletes them from the server.
This works fine for a couple of hours, but then I get an EOLEException saying the Component was busy and got disconnected. After that the service hangs and needs to be restarted.

The wodSFTP-Component is initialized like this:

  sftp.Connect;
  sftp.LicenseKey := 'MyLicenseKey';
  sftp.Hostname := 'MyServer';
  sftp.Login := 'MyUsername';
  sftp.Port := 22;
  sftp.Password := 'MyPassword';
  sftp.Blocking := true;
  sftp.Authentication := authPassword;
  sftp.ProxyType := ProxyNone;

The code that executes every 15 minutes looks like this

try
  sftp.Connect1;
  sftp.RemotePath := '/files';
  sftp.LocalPath := 'c:\temp\download\';
  sftp.GetFiles(sftp.LocalPath, sftp.RemotePath, 1);
  RemoveFilesFromServer; // Deletes only the downloaded files
  sftp.Disconnect1;
except
  on ex: Exception do
  begin
    fLastError := Now;
    bugreport := CreateBugReport(etNormal, ex);
    AddLog(bugreport);
  end;
end;

If another error occurs (e.g. wrong authentication) the error is logged, but the service continues, which is the desired behaviour.

Why is the behaviour different after the 'Component was busy...'-error?

Best Regards,
Nils

EDIT: The Exception occurs during the sftp.GetFiles call.

Hi.

I am not sure why would 'component busy' error be different than any other error we're returning, so I'm confused on why you cannot catch it. Do you have idea in which circum stances this happens? Does it all happen in the same thread where wodSFTP is running at?

Any chance we could duplicate this somehow to try to determine where/why it happens?

Kreso

Hi,

Thanks for your fast reply.

I am not sure why would 'component busy' error be different than any other error we're returning, so I'm confused on why you cannot catch it.

The exception is catched and logged successfully, but it does not continue after.

Do you have idea in which circum stances this happens?

Not really, just that it happens between 6 and 9 hours of program uptime and that there were no files on the sftp server which could be downloaded (we get new files usually 2 or 3 times a day, the rest of the time there are no files available)

Does it all happen in the same thread where wodSFTP is running at?

The application has just the main thread.

I'm currently running the service on another pc with a local sftp server. This setup is working fine for almost 24 hours now. I'll change the setup so that it connects to the remote sftp server and see if I can find out any differences.

Nils

Nils,

when you say app doesn't continue - what exactly happens in your code after the exception? Do you reconnect?

Did you try to take new instance of wodSFTP and re-try with it?

Kreso

when you say app doesn't continue - what exactly happens in your code after the exception? Do you reconnect?

After the exception is logged to a textfile the procedure is finished.
There is a TTimer that runs every couple of seconds and checks if the time since the last connect is greater than 15 minutes, which then calls the procedure.

procedure TdmTransfer.tiCheckTimer(Sender: TObject);
var
  dt: TDateTime;
begin
  dt := Now;
  if MinuteSpan(fLastCheck, dt) >= 15 then
  begin
    fLastCheck := dt;
    CheckServerForNewFiles;
  end;
end;

Did you try to take new instance of wodSFTP and re-try with it?

No, I use the same instance of wodSFTP through the entire life time of the program.
I'll check this out and let you know if this works

Hi.

And you're saying that when 'Component was busy' exception is thrown, your timer never executes again?

Kreso

that is correct.

I also found out, that after the exception is thrown, the wodSFTP component loses it's LicenseKey.

Right now I solved the problem by creating a new instance of wodSFTP every time I check for new files. works good so far.

Hi.

If it works I will not complain. But it's very strange to lose the license key - this cannot happen. Perhaps way you're using it is somehow strange. Best thing I could do is try to duplicate it on our PC to see what could be going on.

Kreso

Hi,
it happened again. this time after 4 days and 3 hours of program uptime.

The Exception was thrown by sftp.GetFiles(...) and was handled but then the application stopped.

I don't know what to try at this point.

I attached the current source code:
-CheckServerForNewFiles is called every minute by a TTimer
-sftpLoopItem is the event of the wodSFTP
-RemoveFilesFromServer deletes every remote file that was not skipped by sftpLoopItem

these methods are defined in a single TDataModule which is created when the service application is started and freed when the service is stopped.

procedure TdmTransfer.CheckServerForNewFiles;
var
  filenames: TStringDynArray;
  bugreport: string;
  fn: string;
  sftp: TwodSFTPCom;
begin
  try
    fFilesToBeRemoved.Clear;
    CoInitialize(nil);
    sftp := TwodSFTPCom.Create(self);
    sftp.Connect;
    sftp.LicenseKey := 'myLicenseKey';
    sftp.Hostname := 'myHostname';
    sftp.Login := 'myUsername';
    sftp.Port := 22;
    sftp.Password := 'myPassword';
    sftp.Blocking := true;
    sftp.Authentication := authPassword;
    sftp.ProxyType := ProxyNone;
    sftp.OnLoopItem := sftpLoopItem;

    sftp.Connect1;
    sftp.RemotePath := TSettings.Instance.RemotePath;
    sftp.LocalPath := 'c:\temp\download\';
    sftp.GetFiles(sftp.LocalPath, sftp.RemotePath, 1);

    filenames := TDirectory.GetFiles('c:\temp\download\', '*.XML');
    if length(filenames) > 0 then
    begin
      MoveFilesToImport(filenames);
      RemoveFilesFromServer(sftp);
    end
    else
      TLogger.AddLog('No files found');
    sftp.Disconnect1;
    sftp.Free;
    CoUninitialize;
  except
    on ex: Exception do
    begin
      bugreport := CreateBugReport(etNormal, ex);
      AddLog(bugreport);
    end;
  end;
end;

procedure TdmTransfer.RemoveFilesFromServer(aSFTP: TwodSFTPCom);
var
  fn: string;
begin
  TLogger.AddLog('Deleting files from server');
  for fn in fFilesToBeRemoved do
  begin
    TLogger.AddLog('  ' + fn);
    aSFTP.DeleteFile(fn);
  end;
  fFilesToBeRemoved.Clear;
end;

procedure TdmTransfer.sftpLoopItem(ASender: TObject; var LocalFile, RemoteFile: WideString; ItemType: TOleEnum;
  var Skip: WordBool);
begin
  Skip := pos('.XML', string(RemoteFile)) <= 0;
  if not Skip then
    fFilesToBeRemoved.Add(RemoteFile);
end;

I found out that after the EOleException, there is an access violation inside the wodSFTP.dll.

This was written to the Windows EventLog: (translated from german)
Service failed during execute:Accessviolation at Address 6701E5C8 in Module 'wodSFTP.dll'. Read from Address 000008B8

Hi.

I would need to run this here in development mode to see what's going on. Any chance I can do it, to have as much same environment as yours? Can you compile it perhaps as standalone desktop EXE?

BTW I assume you're using latest version of wodSFTP?

Kreso