Testing TIdHTTPServer in debug mode Win64 issue

This is the forum for miscellaneous technical/programming questions.

Moderator: 2ffat

Testing TIdHTTPServer in debug mode Win64 issue

Postby Ahmed Sayed » Thu Nov 08, 2018 4:24 pm

I am trying to test a simple server that does nothing actually the problem is when i try test the server in debug mode using TIdHTTPServer. I am doing a stress test using a multi-threaded client when i test with requests between 10 and 50. each request is in its own thread i don't get the error often which is " Error sending data: (12029) A connection with the server could not be established" but when i create for example 100 thread in one shot i get this error for about 50% of the requests made by the client.

When i make the same test but without debug mode every thing works well. So is this a bug or am i doing something wrong i use C++ Builder Berlin 10.1 Update 2.

I tested this with 3 client components TNetHTTPClient - TRESTClient - TIdHTTP
and all 3 gave me the same results.

Here is my client code:

Code: Select all
//---------------------------------------------------------------------------

#include <vcl.h>
#pragma hdrstop

#include "MainU.h"
//---------------------------------------------------------------------------
#pragma package(smart_init)
#pragma resource "*.dfm"
TMain *Main;
//---------------------------------------------------------------------------
__fastcall TMain::TMain(TComponent* Owner)
   : TForm(Owner)
{
TestTime->Date = Now();
TestTime->Time = Now();
ResetCounter();
}
//---------------------------------------------------------------------------
void TMain::ResetCounter()
{
ExecutedCount = 0;
SuccessCount = 0;
ErrorsCount = 0;
ExceptionsCount = 0;

Executed->Caption = "Executed: 0";
Success->Caption = "Success: 0";
Errors->Caption = "Errors: 0";
Exceptions->Caption = "Exceptions: 0";
}
//---------------------------------------------------------------------------
void TMain::Execute(String aFramework, String ID)
{
if (aFramework == "Net")
   Thread(FuncBind( &TMain::NetExecute, this,ID));

if (aFramework == "REST")
   Thread(FuncBind( &TMain::RESTExecute, this,ID));

if (aFramework == "Indy")
   Thread(FuncBind( &TMain::IndyExecute, this, ID));
}
//---------------------------------------------------------------------------
void TMain::NetExecute(String ID)
{
if (Headers->Strings->Text.Trim() == "=")
   Headers->Strings->Clear();

unique_ptr<TNetHTTPClient> HTTPClient(new TNetHTTPClient(nullptr));

String Response;
try
   {
   HTTPClient->CustomHeaders["Connection"] = "close";
   HTTPClient->HandleRedirects = false;
   HTTPClient->ConnectionTimeout = TimeOut->Value;
   HTTPClient->ResponseTimeout = TimeOut->Value;


   if (Headers->Strings->Count > 0)
      {
      for (int i = 0; i < Headers->Strings->Count; i++)
         HTTPClient->CustomHeaders[Headers->Strings->Names[i]] = Headers->Strings->ValueFromIndex[i];
      }

   AddCount(1);
   TStopwatch Timer = TStopwatch::Create();
   Timer.Reset();
   Timer.Start();

   _di_IHTTPResponse AResponse = HTTPClient->Execute(Method->Text, URL->Text);
   Timer.Stop();
   String Timing = Timer.ElapsedMilliseconds / 1000.0;

   Response = AResponse->ContentAsString();

   if (AResponse->StatusCode == 200)
      AddCount(2);
   else
      {
      AddCount(3);
      ShowError(ID, String(AResponse->StatusCode) + ": " + AResponse->StatusText + " - " + Timing + "\n" + Response);
      }
   }
catch (const Exception &E)
   {
   AddCount(4);
   ShowError(ID,"Exception " + E.Message + " - " + Response);
    }
}
//---------------------------------------------------------------------------
void TMain::RESTExecute(String ID)
{
if (Headers->Strings->Text.Trim() == "=")
   Headers->Strings->Clear();

unique_ptr<TRESTClient> RESTClient(new TRESTClient(nullptr));
unique_ptr<TRESTRequest> RESTRequest(new TRESTRequest(nullptr));
unique_ptr<TRESTResponse> RESTResponse(new TRESTResponse(nullptr));

try
   {
   RESTClient->HandleRedirects = false;
   RESTClient->BaseURL = URL->Text;
   RESTRequest->Client = RESTClient.get();
   RESTRequest->Response = RESTResponse.get();
   RESTRequest->Timeout = TimeOut->Value;

   RESTRequest->AddParameter("Connection","close", pkHTTPHEADER);

   if (Headers->Strings->Count > 0)
      {
      for (int i = 0; i < Headers->Strings->Count; i++)
         {
         RESTRequest->AddParameter(Headers->Strings->Names[i],Headers->Strings->ValueFromIndex[i], pkHTTPHEADER);
         }
      }

   RESTRequest->Method = (TRESTRequestMethod)GetEnumValue(__delphirtti(TRESTRequestMethod), "rm" + Method->Text);
   AddCount(1);
   TStopwatch Timer = TStopwatch::Create();
   Timer.Reset();
   Timer.Start();

   RESTRequest->Execute();
   Timer.Stop();
   String Timing = Timer.ElapsedMilliseconds / 1000.0;

   if (RESTResponse->StatusCode == 200)
      AddCount(2);
   else
      {
      AddCount(3);
      ShowError(ID, String(RESTResponse->StatusCode) + ": " + RESTResponse->StatusText + " - " + Timing + "\n" + RESTResponse->Content);
      }
   }
catch (const Exception &E)
   {
   AddCount(4);
   ShowError(ID,"Exception " + E.Message + " - " + RESTResponse->Content);
   }
}
//---------------------------------------------------------------------------
void TMain::IndyExecute(String ID)
{
if (Headers->Strings->Text.Trim() == "=")
   Headers->Strings->Clear();

unique_ptr<TIdHTTP> IdHTTP(new TIdHTTP(nullptr));

String Response;
try
   {
   IdHTTP->ConnectTimeout = TimeOut->Value;
   IdHTTP->ReadTimeout = TimeOut->Value;

   IdHTTP->Request->Connection = "close";
   if (Headers->Strings->Count > 0)
      {
      for (int i = 0; i < Headers->Strings->Count; i++)
         {
         IdHTTP->Request->RawHeaders->Values[Headers->Strings->Names[i]] = Headers->Strings->ValueFromIndex[i];
         }
      }

   AddCount(1);
   TStopwatch Timer = TStopwatch::Create();
   Timer.Reset();
   Timer.Start();

   if (Method->Text == "GET")
      Response = IdHTTP->Get(URL->Text);

   if (Method->Text == "POST")
      Response = IdHTTP->Post(URL->Text,"");

   if (Method->Text == "PUT")
      Response = IdHTTP->Put(URL->Text, nullptr);

   if (Method->Text == "DELETE")
      Response = IdHTTP->Delete(URL->Text);

   Timer.Stop();
   String Timing = Timer.ElapsedMilliseconds / 1000.0;


   if (IdHTTP->ResponseCode == 200)
      AddCount(2);
   else
      {
      AddCount(3);
      ShowError(ID, String(IdHTTP->ResponseCode) + ": " + IdHTTP->ResponseText + " - " + Timing + "\n" + Response);
      }
   }
catch (const Exception &E)
   {
   AddCount(4);
   ShowError(ID,"Exception " + E.Message + " - " + Response);
   }
}
//---------------------------------------------------------------------------
void TMain::DoAddCount(int Index)
{
switch (Index)
   {
   case 1 : ExecutedCount++; break;
   case 2 : SuccessCount++; break;
   case 3 : ErrorsCount++; break;
   case 4 : ExceptionsCount++; break;

   default: break;
   }

Executed->Caption = "Executed: " + String(ExecutedCount);
Success->Caption = "Success: " + String(SuccessCount);
Errors->Caption = "Errors: " + String(ErrorsCount);
Exceptions->Caption = "Exceptions: " + String(ExceptionsCount);
}
//---------------------------------------------------------------------------
void TMain::AddCount(int Index)
{
NotifyUI(FuncBind( &TMain::DoAddCount, this, Index));
}
//---------------------------------------------------------------------------
void TMain::DoShowError(String ID, String Text)
{
Results->Lines->Add("----------------------");
Results->Lines->Add(ID + "- " + Text);
Results->Lines->Add("----------------------");
}
//---------------------------------------------------------------------------
void TMain::ShowError(String ID, String Text)
{
NotifyUI(FuncBind( &TMain::DoShowError, this, ID, Text));
}
//---------------------------------------------------------------------------
void __fastcall TMain::SendClick(TObject *Sender)
{
ResetCounter();
Results->Clear();
Execute(Framework->Text, 1);
}
//---------------------------------------------------------------------------
void TMain::DoRunTest()
{
for (int i = 0; i < ThreadsCount->Value; i++)
   {
   Execute(Framework->Text,i+1);

   if (Interval->Value > 0)
      Sleep(Interval->Value);
   }
}
//---------------------------------------------------------------------------
//---------------------------------------------------------------------------
void TMain::RunTest()
{
StartTest->Enabled = false;
Results->Clear();

ResetCounter();

ThreadWait(FuncBind( &TMain::DoRunTest, this));

StartTest->Enabled = true;
}
//---------------------------------------------------------------------------
void __fastcall TMain::StartTestClick(TObject *Sender)
{
RunTest();
}
//---------------------------------------------------------------------------
void __fastcall TMain::TestTimerTimer(TObject *Sender)
{
Results->Text = Time();

if (Now() >= TestTime->Time)
   {
    TestTimer->Enabled = false;
   RunTest();
   }
}
//---------------------------------------------------------------------------
void __fastcall TMain::StartTimedClick(TObject *Sender)
{
TestTimer->Enabled = true;
}
//---------------------------------------------------------------------------


Appreciate any help. Thanks in advance

Regards,
Ahmed Sayed
Ahmed Sayed
 
Posts: 4
Joined: Thu Nov 08, 2018 4:12 pm

Re: Testing TIdHTTPServer in debug mode Win64 issue

Postby rlebeau » Fri Nov 09, 2018 1:31 pm

Ahmed Sayed wrote:I am trying to test a simple server that does nothing


Well, do nothing or not, you didn't provide ANY details about how you setup the server, or what its code looks like. Since all three client libraries are failing, the problem has to be on the server side.

Ahmed Sayed wrote:" Error sending data: (12029) A connection with the server could not be established"


That is the WinHTTP library's ERROR_WINHTTP_CANNOT_CONNECT error code.

Ahmed Sayed wrote:but when i create for example 100 thread in one shot i get this error for about 50% of the requests made by the client.


You are likely overwhelming the server's backlog of pending connections. In fact, TIdHTTPServer's default backlog is just 15 (configurable via the ListenQueue property). That does not mean the server is limited to 15 connected clients max (that is configurable via the MaxConnections property instead). It just means the server's listening socket can hold only up to 15 (by default) pending connections at a time, waiting to be accepted by the server app. If a new client tries to connect and the backlog is full, that connection gets forcibly dropped, at least on Windows, anyway:

The listen function is typically used by servers that can have more than one connection request at a time. If a connection request arrives and the queue is full, the client will receive an error with an indication of WSAECONNREFUSED.


Other platforms, like Linux, may behave differently, in an attempt to slightly delay the connection so the backlog has time to free up room to service the connection without dropping it completely (though that may still happen).

Ahmed Sayed wrote:When i make the same test but without debug mode every thing works well.


Debug vs Release mode has nothing to do with this issue.

Ahmed Sayed wrote:Here is my client code:


Just an FYI, in your TIdHTTP usage, you should enable the hoNoProtocolErrorException flag in the TIdHTTP::HTTPOptions property, otherwise HTTP layer errors will be handled by your ExceptionsCount and not your ErrorsCount. By default, TIdHTTP throws an EIdHTTPProtocolException when it receives an HTTP error code from the server, and thus your code will not reach the 'if (IdHTTP->ResponseCode == 200)' check.

Also, DO NOT populate the TIdHTTP::Request::RawHeaders property directly. You need to use the TIdHTTP::Request::CustomHeaders property instead. TIdHTTP clears and repopulates the RawHeaders whenever it is preparing a new request.
Remy Lebeau (TeamB)
Lebeau Software
User avatar
rlebeau
BCBJ Author
BCBJ Author
 
Posts: 1544
Joined: Wed Jun 01, 2005 3:21 am
Location: California, USA

Re: Testing TIdHTTPServer in debug mode Win64 issue

Postby Ahmed Sayed » Sun Nov 11, 2018 3:20 am

Thanks Remy,

But what i meant with debug mode is I am running the server with the debugger attached (F9). Which makes the server a slower than the running without debugger. Also the server does nothing that serve anything major because all i do is on the server side:

Code: Select all
//---------------------------------------------------------------------------
#include <vcl.h>
#pragma hdrstop

#include "MainU.h"
//---------------------------------------------------------------------------
#pragma package(smart_init)
#pragma resource "*.dfm"
TMain *Main;
//---------------------------------------------------------------------------
__fastcall TMain::TMain(TComponent* Owner)
   : TForm(Owner)
{

}
//---------------------------------------------------------------------------
void __fastcall TMain::ConnectedClick(TObject *Sender)
{
HTTPServer->DefaultPort = Port->Text.ToInt();
HTTPServer->Active = Connected->Checked;
}
//---------------------------------------------------------------------------
void TMain::AddRequesSection(TStringList *Strings,String Key, String Value)
{
Strings->Add(Key + ":-");
Strings->Add(StringOfChar('-',Key.Length()));
Strings->Add(Value.Trim());
Strings->Add("//-------------------------------------//");
}
//---------------------------------------------------------------------------
void __fastcall TMain::HTTPServerCommandGet(TIdContext *AContext, TIdHTTPRequestInfo *ARequestInfo,
        TIdHTTPResponseInfo *AResponseInfo)
{
unique_ptr<TStringList> RequestDetails(new TStringList);
RequestDetails->NameValueSeparator = ':';
RequestDetails->TrailingLineBreak = false;
RequestDetails->Clear();

AddRequesSection(RequestDetails.get(), "RawHTTPCommand",ARequestInfo->RawHTTPCommand);

unique_ptr<TStringList> Headers(new TStringList);
ARequestInfo->RawHeaders->ConvertToStdValues(Headers.get());
AddRequesSection(RequestDetails.get(), "RawHeaders", Headers->Text);
Headers->Clear();
ARequestInfo->CustomHeaders->ConvertToStdValues(Headers.get());
AddRequesSection(RequestDetails.get(), "CustomHeaders", Headers->Text);

AddRequesSection(RequestDetails.get(), "Command", ARequestInfo->Command);
AddRequesSection(RequestDetails.get(), "Document", ARequestInfo->Document);
AddRequesSection(RequestDetails.get(), "URI", ARequestInfo->URI);

AddRequesSection(RequestDetails.get(), "RemoteIP", ARequestInfo->RemoteIP);
AddRequesSection(RequestDetails.get(), "Host", ARequestInfo->Host);
AddRequesSection(RequestDetails.get(), "From", ARequestInfo->From);
AddRequesSection(RequestDetails.get(), "Username", ARequestInfo->Username);
AddRequesSection(RequestDetails.get(), "Password", ARequestInfo->Password);
AddRequesSection(RequestDetails.get(), "AuthExists", BoolToStr(ARequestInfo->AuthExists,true));
AddRequesSection(RequestDetails.get(), "BasicAuthentication", BoolToStr(ARequestInfo->BasicAuthentication,true));
AddRequesSection(RequestDetails.get(), "AuthUsername", ARequestInfo->AuthUsername);
AddRequesSection(RequestDetails.get(), "AuthPassword", ARequestInfo->AuthPassword);
AddRequesSection(RequestDetails.get(), "Referer", ARequestInfo->Referer);
AddRequesSection(RequestDetails.get(), "UserAgent", ARequestInfo->UserAgent);
AddRequesSection(RequestDetails.get(), "ProxyConnection", ARequestInfo->ProxyConnection);

AddRequesSection(RequestDetails.get(), "Params", ARequestInfo->Params->Text);
AddRequesSection(RequestDetails.get(), "UnparsedParams", ARequestInfo->UnparsedParams);
AddRequesSection(RequestDetails.get(), "FormParams", ARequestInfo->FormParams);
AddRequesSection(RequestDetails.get(), "QueryParams", ARequestInfo->QueryParams);
AddRequesSection(RequestDetails.get(), "Version", ARequestInfo->Version);
AddRequesSection(RequestDetails.get(), "VersionMajor", ARequestInfo->VersionMajor);
AddRequesSection(RequestDetails.get(), "VersionMinor", ARequestInfo->VersionMinor);
AddRequesSection(RequestDetails.get(), "Accept", ARequestInfo->Accept);
AddRequesSection(RequestDetails.get(), "AcceptCharSet", ARequestInfo->AcceptCharSet);
AddRequesSection(RequestDetails.get(), "AcceptEncoding", ARequestInfo->AcceptEncoding);
AddRequesSection(RequestDetails.get(), "AcceptLanguage", ARequestInfo->AcceptLanguage);

AddRequesSection(RequestDetails.get(), "HasContentRange", BoolToStr(ARequestInfo->HasContentRange,true));
AddRequesSection(RequestDetails.get(), "Range", ARequestInfo->Range);
AddRequesSection(RequestDetails.get(), "Ranges", ARequestInfo->Ranges->Text);
AddRequesSection(RequestDetails.get(), "ContentRangeUnits", ARequestInfo->ContentRangeUnits);
AddRequesSection(RequestDetails.get(), "ContentRangeInstanceLength", ARequestInfo->ContentRangeInstanceLength);
AddRequesSection(RequestDetails.get(), "ContentRangeStart", ARequestInfo->ContentRangeStart);
AddRequesSection(RequestDetails.get(), "ContentRangeEnd", ARequestInfo->ContentRangeEnd);
AddRequesSection(RequestDetails.get(), "MethodOverride", ARequestInfo->MethodOverride);
AddRequesSection(RequestDetails.get(), "HasContentRangeInstance", BoolToStr(ARequestInfo->HasContentRangeInstance,true));
AddRequesSection(RequestDetails.get(), "CacheControl", ARequestInfo->CacheControl);
AddRequesSection(RequestDetails.get(), "CharSet", ARequestInfo->CharSet);
AddRequesSection(RequestDetails.get(), "Connection", ARequestInfo->Connection);

AddRequesSection(RequestDetails.get(), "Date", ARequestInfo->Date.DateTimeString());
AddRequesSection(RequestDetails.get(), "ETag", ARequestInfo->ETag);
AddRequesSection(RequestDetails.get(), "Expires", ARequestInfo->Expires.DateTimeString());
AddRequesSection(RequestDetails.get(), "LastModified", ARequestInfo->LastModified.DateTimeString());
AddRequesSection(RequestDetails.get(), "Pragma", ARequestInfo->Pragma);
AddRequesSection(RequestDetails.get(), "TransferEncoding", ARequestInfo->TransferEncoding);

AddRequesSection(RequestDetails.get(), "HasContentLength", BoolToStr(ARequestInfo->HasContentLength,true));
AddRequesSection(RequestDetails.get(), "ContentDisposition", ARequestInfo->ContentDisposition);
AddRequesSection(RequestDetails.get(), "ContentEncoding", ARequestInfo->ContentEncoding);
AddRequesSection(RequestDetails.get(), "ContentLanguage", ARequestInfo->ContentLanguage);
AddRequesSection(RequestDetails.get(), "ContentLength", ARequestInfo->ContentLength);
AddRequesSection(RequestDetails.get(), "ContentType", ARequestInfo->ContentType);
AddRequesSection(RequestDetails.get(), "ContentVersion", ARequestInfo->ContentVersion);

String PostStr = "No Post Stream";

if (ARequestInfo->ContentLength > 0)
   {
   if (ARequestInfo->PostStream != nullptr)
      {
      ARequestInfo->PostStream->Position = 0;
      PostStr = ReadStringFromStream(ARequestInfo->PostStream,-1);
        }
   }
AddRequesSection(RequestDetails.get(), "PostStream", PostStr);

AResponseInfo->ContentText = RequestDetails->Text.Trim();
}
//---------------------------------------------------------------------------
void __fastcall TMain::FormClose(TObject *Sender, TCloseAction &Action)
{
HTTPServer->Active = false;
}
//---------------------------------------------------------------------------


So, i am not connecting to a database or something that needs a lot of processing
it is like the IDE is the one that refuses the connection.

Also if there was anything wrong on the server side then it provided me with a proper error or exception stating what the issue is or where it is. That's why i am running the app with debugger and it does not give me any exceptions at all when i run the server that way only the errors i mentioned on the client side.

Thanks Again
Ahmed Sayed
 
Posts: 4
Joined: Thu Nov 08, 2018 4:12 pm

Re: Testing TIdHTTPServer in debug mode Win64 issue

Postby rlebeau » Mon Nov 12, 2018 12:11 pm

Ahmed Sayed wrote:But what i meant with debug mode is I am running the server with the debugger attached (F9).


I know what you meant.

Ahmed Sayed wrote:Which makes the server a slower than the running without debugger.


Not really relevant to your issue. The speed in which the server processes requests is not related to the speed in which it accepts new socket connections. They are handled on separate threads. Though, it does make sense that debugging would be a little slower, since the IDE has to keep track of all running threads, receive thread names from Indy and associate them which entries in that list, etc. But I don't see how that could possibly slow down the the listening threads that are accepting clients and spawning new threads for each one.

Ahmed Sayed wrote:
Code: Select all
void __fastcall TMain::ConnectedClick(TObject *Sender)
{
HTTPServer->DefaultPort = Port->Text.ToInt();
HTTPServer->Active = Connected->Checked;
}


Just an FYI, if you activate the server, then deactivate it, then change the DefaultPort, and reactivate, the new DefaultPort will not take effect unless you clear/reset the server's Bindings collection, which you are not doing.

Ahmed Sayed wrote:
Code: Select all
ARequestInfo->CustomHeaders->ConvertToStdValues(Headers.get());
AddRequesSection(RequestDetails.get(), "CustomHeaders", Headers->Text);


FYI, CustomHeaders is always blank in a request. It is only used when making a response.

Ahmed Sayed wrote:it is like the IDE is the one that refuses the connection.


It is not.

Did you try increasing the value of the server's ListenQueue property, like I suggested earlier? The default of 15 is simply too low for your level of stress testing. If you try to connect 100 clients at the same time, they are likely going to overwhelm the server's backlog. Whether you use debugging or not.
Remy Lebeau (TeamB)
Lebeau Software
User avatar
rlebeau
BCBJ Author
BCBJ Author
 
Posts: 1544
Joined: Wed Jun 01, 2005 3:21 am
Location: California, USA

Re: Testing TIdHTTPServer in debug mode Win64 issue

Postby Ahmed Sayed » Tue Nov 13, 2018 3:35 am

I will try increasing the ListenQueue as you said but i am talking about service that might receive 100,000 requests at the same time. What is the appropriate amount for any thing not just ListenQueue property. Also, is TIdHTTPServer will be up to the task to handle that big number of requests or do i have to use some other component.
Ahmed Sayed
 
Posts: 4
Joined: Thu Nov 08, 2018 4:12 pm

Re: Testing TIdHTTPServer in debug mode Win64 issue

Postby rlebeau » Tue Nov 13, 2018 1:06 pm

Ahmed Sayed wrote:i am talking about service that might receive 100,000 requests at the same time.


You are likely going to run into trouble trying to handle that much traffic on one server alone. You should consider using multiple servers with a load balancer in front of them.
Remy Lebeau (TeamB)
Lebeau Software
User avatar
rlebeau
BCBJ Author
BCBJ Author
 
Posts: 1544
Joined: Wed Jun 01, 2005 3:21 am
Location: California, USA

Re: Testing TIdHTTPServer in debug mode Win64 issue

Postby Ahmed Sayed » Wed Nov 14, 2018 3:32 am

Yes that's what i am talking about is there a formula that can use or an equation that will give me the required number of servers and how much CPU speed i need for each server? Also, is there a maximum number of requests no matter how powerful the server is that Indy can handle?
Ahmed Sayed
 
Posts: 4
Joined: Thu Nov 08, 2018 4:12 pm


Return to Technical

Who is online

Users browsing this forum: Bing [Bot] and 3 guests