sockets die after a while (wodHttpDLX)
by Klinzter, Tuesday, September 10, 2013, 06:48 (4093 days ago)
Hello,
I am running about 10 sockets to download a list of images from different domain names and sometimes the connection times out and I know how to handle like move to next.. or the domain does not exists anymore and also move to next..
I have set DebugFile = debug_ & Index & .txt
so that I can know at the end when im left with just 1 socket to see what happened.. and the last line that I see is this..
** Connecting to www.site.com on port 80
Disconnected: ErrorCode 0
that's it nothing else but no action gets triggered so I lose that socket..
what can I do to prevent idle sockets ?
Re: sockets die after a while
by wodSupport, Tuesday, September 10, 2013, 09:57 (4092 days ago) @ Klinzter
Klinzter hi,
It is really hard to say because I'm not sure that I clearly understand what you need.
Anyway, from my understanding you need to close connection, right? Maybe you can manually call Disconnect Method after some time and socket will be closed. Can you try that?
Re: sockets die after a while
by Klinzter, Thursday, September 12, 2013, 04:54 (4091 days ago) @ wodSupport
the issue is that it does not timeout and or gets disconnected..
all I see when I check the debug file for (dead sockets) is this..
** Connecting to www.site.com on port 80
I dont know if this is the case but for example.. lets say I have 10 sockets running downloading images from random urls..
socket5 goes to page (www.site1.com/image1.jpg) and downloads image and gets server response 200 and response length (1936565) from server.. and finishes downloading image
then again socket5 goes to next url (www.site2.com/image2.jpg) and tries to connect but it doesn't connect correctly or something happens and I see in the disconnect event the same response 200 and length (1936565) from server
so something is wrong with the wrapper not clearing the response body data, length, etc.. before connect, can you confirm this by looking at the source code?
The only way for now to get it to work is by calling this every time before new connection
Set wodHttp1 = New wodHttpDLXCom
Set wodHttp1.Notification = Me
Re: sockets die after a while
by Klinzter, Thursday, September 12, 2013, 06:19 (4091 days ago) @ Klinzter
additional information..
I've been doing some code research and I normally use debug everywhere so by placing this in the disconnected event
Debug.Print( dead: & Owner.Tag & - Case: & iRequest(Owner.Tag) & - Date: & Date.Now().ToLongTimeString & - ErrorCode: & ErrorCode & - Length: & Owner.Response.Length & - & wodHttp1(Owner.Tag).Response.Length)
I get this >
dead: 3 - Case: 3 - Date: 12:14:19 AM - ErrorCode: 0 - Length: 10493931 - 10493931
dead: 3 - Case: 3 - Date: 12:14:20 AM - ErrorCode: 0 - Length: 10493931 - 10493931
dead: 2 - Case: 2 - Date: 12:14:20 AM - ErrorCode: 10054 - Length: 12427916 - 12427916
dead: 2 - Case: 3 - Date: 12:14:21 AM - ErrorCode: 0 - Length: 12427916 - 12427916
dead: 2 - Case: 3 - Date: 12:14:21 AM - ErrorCode: 0 - Length: 12427916 - 12427916
dead: 3 - Case: 2 - Date: 12:14:27 AM - ErrorCode: 0 - Length: 1209712 - 1209712
dead: 3 - Case: 2 - Date: 12:14:27 AM - ErrorCode: 0 - Length: 1209712 - 1209712
this is wrong i think.. so next test is good!
it looks like its working all fine when calling this right before connecting and see the lenght for the responses now..
wodHttp1(Index) = New wodHttpDLXCom
wodHttp1(Index).Notification = Me
wodHttp1(Index).Tag = Index
wodHttp1(Index).AutoRedirect = True
wodHttp1(Index).Timeout = 30
dead: 3 - Case: 2 - Date: 12:11:33 AM - ErrorCode: 10054 - Length: 42337320 - 0
dead: 3 - Case: 3 - Date: 12:11:34 AM - ErrorCode: 0 - Length: 0 - 0
dead: 1 - Case: 2 - Date: 12:11:34 AM - ErrorCode: 10054 - Length: 16415969 - 0
dead: 3 - Case: 3 - Date: 12:11:34 AM - ErrorCode: 0 - Length: 0 - 0
dead: 1 - Case: 3 - Date: 12:11:34 AM - ErrorCode: 0 - Length: 0 - 0
dead: 3 - Case: 2 - Date: 12:11:34 AM - ErrorCode: 10054 - Length: 17342699 - 0
dead: 1 - Case: 3 - Date: 12:11:34 AM - ErrorCode: 0 - Length: 0 - 0
dead: 1 - Case: 2 - Date: 12:11:35 AM - ErrorCode: 10054 - Length: 43928693 - 0
Re: sockets die after a while
by wodSupport, Thursday, September 12, 2013, 07:16 (4091 days ago) @ Klinzter
Hi.
Can you please provide exact sample for this behavior? I doubt wodHttpDLX doesn't clear our responses, so I think something else is involved here. But I would need to try it for myself.
Thanks!
Kreso
Re: sockets die after a while
by Klinzter, Thursday, September 12, 2013, 07:22 (4091 days ago) @ wodSupport
that's what I thought it was happening but no. I've been testing and its really hard to come across this error I have to leave it running for about 15 minutes to find a dead socket I've been trying 30 different things to make this work and the only one that works is by creating a new socket each time before connect. so this is obviously not an error with my app but with wodHttp.. if you find the issue and get it patched up ill be buying the update because my subscription expired.. if you cant find the issue its okay ill stick to (create new socket) on each new connection.
wodhttp clears responses but after and not before new connection...
Re: sockets die after a while
by wodsupport, Thursday, September 12, 2013, 08:14 (4090 days ago) @ Klinzter
You can also enable DebugFile like
Http1.DebugFile = somefile.txt
and it will dump everyhting it sends/receive. Beware - file may grow really large. This way you can see if and what is received from remote on all your requests.
But again, I must ask for a sample for duplicate this. 1000 issues can be involved in why it returns 200. Perhaps server really returns 200?
Kreso
Re: sockets die after a while
by Klinzter, Thursday, September 12, 2013, 08:46 (4090 days ago) @ wodsupport
I already have that..
call ClearDebug( somefile.txt )
Http1.DebugFile = somefile.txt
right before making a new connection and I see that it does NOT receive anything.. (when i find dead socket error)
there is no other way to explain it to you if you don't want to understand.
as I said before wodHttp does clear before making a new connection it clears after.. I have the source code and I can see it.. what I am asking is for a way to clear before..
Re: sockets die after a while
by wodSupport, Thursday, September 12, 2013, 09:21 (4090 days ago) @ Klinzter
Hi.
I can only suggest you take a new instance for each new connection. Why is that a problem? New socket is used internally anyway.
Kreso
Re: sockets die after a while
by Klinzter, Thursday, September 12, 2013, 10:56 (4090 days ago) @ wodSupport
the issue with creating new socket right before making a new connection will not work for me because some pages require cookies to move on to next page and if I create a new socket I lose those cookies..
and also I have it to change proxies every 10 minutes.. so by doing this I lose the proxy
Re: sockets die after a while
by woddrazen, Thursday, September 12, 2013, 14:21 (4090 days ago) @ Klinzter
Hi,
Did you try to use RemoveAll Method to remove elements before next request? Does that help maybe?
Drazen
Re: sockets die after a while
by Klinzter, Thursday, September 12, 2013, 21:03 (4090 days ago) @ woddrazen
there is no option for .Response.RemoveAll()
but yes I do have .Request.Headers.RemoveAll()
Re: sockets die after a while
by woddrazen, Friday, September 13, 2013, 12:05 (4089 days ago) @ Klinzter
Hi,
Is there any chance you can show us your code snippet?
Drazen
Re: sockets die after a while
by Klinzter, Friday, September 13, 2013, 22:28 (4089 days ago) @ woddrazen
I sent the source samples to the tech support email
