AAFAQ Process /1043092

Enqueued 0 / 1 Scheduled 20 Processing 4 Succeeded 19,433 Failed 1,148 Deleted 0 Awaiting 0

The job is finished. It will be removed automatically 10/10/2026 00:03:22.

// Job ID: fdbfa638-73cd-40db-9d2c-ced7c19fa276
using EP.BackgroundJobs.Engine.Jobs;

var aAFAQJob = Activate<AAFAQJob>();
aAFAQJob.ProcessCustomer(
    null,
    null,
    "1043092",
    FromJson<WebhookCustomer>("{\"AccountNumber\":\"1043092\",\"CustomerName\":null,\"Url\":\"https://oms.aafaq.ae:3080/rest/api/CardDelivery_Status\",\"IPSEventsURL\":null,\"Token\":null,\"SecondaryToken\":null,\"RowsCount\":500,\"Active\":false,\"Type\":\"EP.BackgroundJobs.Engine.BusinessLogic.Implementations.AAFAQEventPusher\",\"TokenUrl\":null,\"ClientID\":\"@EMX$1090$!M\",\"ClientSecret\":\"C@RDN9!ZR$G\",\"Cookie\":null,\"IsLive\":true,\"ProductCode\":null,\"StatusCodes\":null,\"BackDateCount\":\"7\",\"IsEidRequired\":false,\"ExtraUrl\":null,\"Scope\":null,\"SiteId\":\"667\",\"CertPwd\":null,\"enableCertLog\":false,\"userName\":null,\"Password\":null}"),
    "DATA SOURCE=(DESCRIPTION= (ADDRESS= (PROTOCOL=TCP) (HOST=ocprdexavm-fefsa-scan.epgprdb.epgvcn.oraclevcn.com) (PORT=1521)) (CONNECT_DATA= (SERVER=DEDICATED) (SERVICE_NAME=OPSPRD_OPSPDB01.paas.oracle.com) (FAILOVER_MODE= (TYPE=select) (METHOD=basic))));USER ID=BOC_RUN;Password=B0#8nn$C;Pooling=true;Min Pool Size=0;Connection Lifetime=180;Max Pool Size=50;Incr Pool Size=5;");

Parameters

CurrentCulture	`""`
CurrentUICulture	`""`
RetryCount	`2`

State

10/09/2026 00:03:22 (+22.717s) Succeeded

Latency:: 2m 48.901s
Duration:: 22.716s

+ <1ms Processing

Server:: ep-hangfire-svc-55f796dcbf-fpxvr:1
Worker:: d0eee766

+9.558sTotal Number of AWBs: 52
+9.558s-------- Start AWB_NUM: 1100023348967 -------- 
+9.914sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+9.915sHTTP Code:OK
+9.923s-------- End AWB_NUM: 1100023348967 -------- 
+9.923s-------- Start AWB_NUM: 1100023348967 -------- 
+10.172sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+10.172sHTTP Code:OK
+10.187s-------- End AWB_NUM: 1100023348967 -------- 
+10.187s-------- Start AWB_NUM: 1100023348965 -------- 
+10.419sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+10.419sHTTP Code:OK
+10.427s-------- End AWB_NUM: 1100023348965 -------- 
+10.427s-------- Start AWB_NUM: 1100023348965 -------- 
+10.686sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+10.686sHTTP Code:OK
+10.692s-------- End AWB_NUM: 1100023348965 -------- 
+10.693s-------- Start AWB_NUM: 1100023348934 -------- 
+10.887sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+10.887sHTTP Code:OK
+10.894s-------- End AWB_NUM: 1100023348934 -------- 
+10.894s-------- Start AWB_NUM: 1100023348934 -------- 
+11.095sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+11.095sHTTP Code:OK
+11.101s-------- End AWB_NUM: 1100023348934 -------- 
+11.101s-------- Start AWB_NUM: 1100023348966 -------- 
+11.389sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+11.389sHTTP Code:OK
+11.401s-------- End AWB_NUM: 1100023348966 -------- 
+11.401s-------- Start AWB_NUM: 1100023348966 -------- 
+11.746sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+11.746sHTTP Code:OK
+11.753s-------- End AWB_NUM: 1100023348966 -------- 
+11.753s-------- Start AWB_NUM: 1100023348964 -------- 
+11.994sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+11.994sHTTP Code:OK
+12s-------- End AWB_NUM: 1100023348964 -------- 
+12s-------- Start AWB_NUM: 1100023348964 -------- 
+12.240sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+12.240sHTTP Code:OK
+12.246s-------- End AWB_NUM: 1100023348964 -------- 
+12.247s-------- Start AWB_NUM: 1100023348936 -------- 
+12.491sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+12.491sHTTP Code:OK
+12.498s-------- End AWB_NUM: 1100023348936 -------- 
+12.498s-------- Start AWB_NUM: 1100023348936 -------- 
+12.803sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+12.803sHTTP Code:OK
+12.810s-------- End AWB_NUM: 1100023348936 -------- 
+12.810s-------- Start AWB_NUM: 1100023348947 -------- 
+13.077sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+13.077sHTTP Code:OK
+13.084s-------- End AWB_NUM: 1100023348947 -------- 
+13.084s-------- Start AWB_NUM: 1100023348947 -------- 
+13.232sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+13.232sHTTP Code:OK
+13.238s-------- End AWB_NUM: 1100023348947 -------- 
+13.238s-------- Start AWB_NUM: 1100023348943 -------- 
+13.456sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+13.456sHTTP Code:OK
+13.462s-------- End AWB_NUM: 1100023348943 -------- 
+13.462s-------- Start AWB_NUM: 1100023348943 -------- 
+13.594sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+13.594sHTTP Code:OK
+13.599s-------- End AWB_NUM: 1100023348943 -------- 
+13.599s-------- Start AWB_NUM: 1100023348952 -------- 
+13.855sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+13.855sHTTP Code:OK
+13.862s-------- End AWB_NUM: 1100023348952 -------- 
+13.862s-------- Start AWB_NUM: 1100023348952 -------- 
+14.121sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+14.121sHTTP Code:OK
+14.127s-------- End AWB_NUM: 1100023348952 -------- 
+14.127s-------- Start AWB_NUM: 1100023348951 -------- 
+14.353sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+14.353sHTTP Code:OK
+14.359s-------- End AWB_NUM: 1100023348951 -------- 
+14.359s-------- Start AWB_NUM: 1100023348951 -------- 
+14.667sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+14.667sHTTP Code:OK
+14.673s-------- End AWB_NUM: 1100023348951 -------- 
+14.673s-------- Start AWB_NUM: 1100023348950 -------- 
+14.874sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+14.874sHTTP Code:OK
+14.882s-------- End AWB_NUM: 1100023348950 -------- 
+14.882s-------- Start AWB_NUM: 1100023348950 -------- 
+15.169sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+15.169sHTTP Code:OK
+15.183s-------- End AWB_NUM: 1100023348950 -------- 
+15.183s-------- Start AWB_NUM: 1100023348945 -------- 
+15.458sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+15.458sHTTP Code:OK
+15.464s-------- End AWB_NUM: 1100023348945 -------- 
+15.464s-------- Start AWB_NUM: 1100023348945 -------- 
+15.748sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+15.748sHTTP Code:OK
+15.756s-------- End AWB_NUM: 1100023348945 -------- 
+15.756s-------- Start AWB_NUM: 1100023348942 -------- 
+16.079sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+16.079sHTTP Code:OK
+16.086s-------- End AWB_NUM: 1100023348942 -------- 
+16.086s-------- Start AWB_NUM: 1100023348942 -------- 
+16.343sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+16.343sHTTP Code:OK
+16.349s-------- End AWB_NUM: 1100023348942 -------- 
+16.349s-------- Start AWB_NUM: 1100023348939 -------- 
+16.538sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+16.538sHTTP Code:OK
+16.545s-------- End AWB_NUM: 1100023348939 -------- 
+16.545s-------- Start AWB_NUM: 1100023348939 -------- 
+16.803sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+16.803sHTTP Code:OK
+16.809s-------- End AWB_NUM: 1100023348939 -------- 
+16.809s-------- Start AWB_NUM: 1100023348938 -------- 
+17.047sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+17.048sHTTP Code:OK
+17.057s-------- End AWB_NUM: 1100023348938 -------- 
+17.058s-------- Start AWB_NUM: 1100023348938 -------- 
+17.262sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+17.262sHTTP Code:OK
+17.268s-------- End AWB_NUM: 1100023348938 -------- 
+17.268s-------- Start AWB_NUM: 1100023348949 -------- 
+17.523sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+17.523sHTTP Code:OK
+17.530s-------- End AWB_NUM: 1100023348949 -------- 
+17.531s-------- Start AWB_NUM: 1100023348949 -------- 
+17.754sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+17.754sHTTP Code:OK
+17.767s-------- End AWB_NUM: 1100023348949 -------- 
+17.767s-------- Start AWB_NUM: 1100023348963 -------- 
+18.021sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+18.022sHTTP Code:OK
+18.030s-------- End AWB_NUM: 1100023348963 -------- 
+18.030s-------- Start AWB_NUM: 1100023348963 -------- 
+18.235sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+18.235sHTTP Code:OK
+18.241s-------- End AWB_NUM: 1100023348963 -------- 
+18.241s-------- Start AWB_NUM: 1100023348962 -------- 
+18.432sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+18.432sHTTP Code:OK
+18.439s-------- End AWB_NUM: 1100023348962 -------- 
+18.439s-------- Start AWB_NUM: 1100023348962 -------- 
+18.618sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+18.619sHTTP Code:OK
+18.626s-------- End AWB_NUM: 1100023348962 -------- 
+18.626s-------- Start AWB_NUM: 1100023348961 -------- 
+18.907sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+18.907sHTTP Code:OK
+18.915s-------- End AWB_NUM: 1100023348961 -------- 
+18.915s-------- Start AWB_NUM: 1100023348961 -------- 
+19.020sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+19.020sHTTP Code:OK
+19.026s-------- End AWB_NUM: 1100023348961 -------- 
+19.026s-------- Start AWB_NUM: 1100023348948 -------- 
+19.231sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+19.231sHTTP Code:OK
+19.236s-------- End AWB_NUM: 1100023348948 -------- 
+19.237s-------- Start AWB_NUM: 1100023348948 -------- 
+19.467sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+19.467sHTTP Code:OK
+19.478s-------- End AWB_NUM: 1100023348948 -------- 
+19.478s-------- Start AWB_NUM: 1100023348946 -------- 
+19.739sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+19.739sHTTP Code:OK
+19.746s-------- End AWB_NUM: 1100023348946 -------- 
+19.746s-------- Start AWB_NUM: 1100023348946 -------- 
+19.979sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+19.980sHTTP Code:OK
+19.989s-------- End AWB_NUM: 1100023348946 -------- 
+19.989s-------- Start AWB_NUM: 1100023348944 -------- 
+20.091sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+20.091sHTTP Code:OK
+20.098s-------- End AWB_NUM: 1100023348944 -------- 
+20.099s-------- Start AWB_NUM: 1100023348944 -------- 
+20.396sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+20.396sHTTP Code:OK
+20.403s-------- End AWB_NUM: 1100023348944 -------- 
+20.403s-------- Start AWB_NUM: 1100023348941 -------- 
+20.784sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+20.784sHTTP Code:OK
+20.793s-------- End AWB_NUM: 1100023348941 -------- 
+20.793s-------- Start AWB_NUM: 1100023348941 -------- 
+20.933sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+20.933sHTTP Code:OK
+20.941s-------- End AWB_NUM: 1100023348941 -------- 
+20.941s-------- Start AWB_NUM: 1100023348940 -------- 
+21.223sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+21.223sHTTP Code:OK
+21.231s-------- End AWB_NUM: 1100023348940 -------- 
+21.231s-------- Start AWB_NUM: 1100023348940 -------- 
+21.468sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+21.468sHTTP Code:OK
+21.474s-------- End AWB_NUM: 1100023348940 -------- 
+21.474s-------- Start AWB_NUM: 1100023348937 -------- 
+21.802sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+21.802sHTTP Code:OK
+21.808s-------- End AWB_NUM: 1100023348937 -------- 
+21.808s-------- Start AWB_NUM: 1100023348937 -------- 
+22.071sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+22.071sHTTP Code:OK
+22.078s-------- End AWB_NUM: 1100023348937 -------- 
+22.078s-------- Start AWB_NUM: 1100023348935 -------- 
+22.404sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+22.404sHTTP Code:OK
+22.411s-------- End AWB_NUM: 1100023348935 -------- 
+22.411s-------- Start AWB_NUM: 1100023348935 -------- 
+22.711sThe received result is: {"errorCode":null,"errorMsg":null,"status":"Success"}
+22.711sHTTP Code:OK
+22.718s-------- End AWB_NUM: 1100023348935 -------- 

+1m 18ms Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+ <1ms Scheduled

Retry attempt 2 of 3: Connection request timed out

Enqueue at:: 10/09/2026 00:02:49

+15.004s Failed

An exception occurred during performance of the job.

Oracle.ManagedDataAccess.Client.OracleException

Connection request timed out

Oracle.ManagedDataAccess.Client.OracleException (0x80004005): Connection request timed out
   at OracleInternal.ConnectionPool.PoolManager`3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, OracleConnection connRefForCriteria, String affinityInstanceName, Boolean bForceMatch)
   at OracleInternal.ConnectionPool.OraclePoolManager.Get(ConnectionString csWithNewPassword, Boolean bGetForApp, OracleConnection connRefForCriteria, String affinityInstanceName, Boolean bForceMatch)
   at OracleInternal.ConnectionPool.OracleConnectionDispenser`3.Get(ConnectionString cs, PM conPM, ConnectionString pmCS, SecureString securedPassword, SecureString securedProxyPassword, OracleConnection connRefForCriteria)
   at Oracle.ManagedDataAccess.Client.OracleConnection.Open()
   at EP.BackgroundJobs.Engine.BusinessLogic.Implementations.AAFAQEventPusher.Execute(PerformContext context, WebhookCustomer customer, String courierDatabaseConnectionString) in /src/EP.BackgroundJobs.Engine/BusinessLogic/Implementations/AAFAQEventPusher.cs:line 57

+ <1ms Processing

Server:: ep-hangfire-svc-55f796dcbf-fpxvr:1
Worker:: 51f6596d

+45.007s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+ <1ms Scheduled

Retry attempt 1 of 3: Connection request timed out

Enqueue at:: 10/09/2026 00:01:38

+15.001s Failed

An exception occurred during performance of the job.

Oracle.ManagedDataAccess.Client.OracleException

Connection request timed out

Oracle.ManagedDataAccess.Client.OracleException (0x80004005): Connection request timed out
   at OracleInternal.ConnectionPool.PoolManager`3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, OracleConnection connRefForCriteria, String affinityInstanceName, Boolean bForceMatch)
   at OracleInternal.ConnectionPool.OraclePoolManager.Get(ConnectionString csWithNewPassword, Boolean bGetForApp, OracleConnection connRefForCriteria, String affinityInstanceName, Boolean bForceMatch)
   at OracleInternal.ConnectionPool.OracleConnectionDispenser`3.Get(ConnectionString cs, PM conPM, ConnectionString pmCS, SecureString securedPassword, SecureString securedProxyPassword, OracleConnection connRefForCriteria)
   at Oracle.ManagedDataAccess.Client.OracleConnection.Open()
   at EP.BackgroundJobs.Engine.BusinessLogic.Implementations.AAFAQEventPusher.Execute(PerformContext context, WebhookCustomer customer, String courierDatabaseConnectionString) in /src/EP.BackgroundJobs.Engine/BusinessLogic/Implementations/AAFAQEventPusher.cs:line 57

+30.013s Processing

Server:: ep-hangfire-svc-55f796dcbf-fpxvr:1
Worker:: 9eb73240

+3.855s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+ <1ms Scheduled

Enqueue at:: 10/09/2026 00:00:10

Parameters

Requeue Delete State

10/09/2026 00:03:22 (+22.717s) Succeeded

+ <1ms Processing

+1m 18ms Enqueued

+ <1ms Scheduled

+15.004s Failed

Oracle.ManagedDataAccess.Client.OracleException

+ <1ms Processing

+45.007s Enqueued

+ <1ms Scheduled

+15.001s Failed

Oracle.ManagedDataAccess.Client.OracleException

+30.013s Processing

+3.855s Enqueued

+ <1ms Scheduled

10/09/2026 00:00:10 Created

State