Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[opt](s3client) Set ClientConfiguration.requestTimeoutMs default value from 3000ms to 30000ms #49315

Merged
merged 1 commit into from
Mar 22, 2025

Conversation

SWJTU-ZhangLei
Copy link
Contributor

@SWJTU-ZhangLei SWJTU-ZhangLei commented Mar 20, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

… 3000ms to 30000ms

* ClientConfiguration.requestTimeoutMs default is 3000ms in `aws-sdk-cpp`, this should be more than adequate for most services.
  However, we meet problem when cloud provider's storage service or network not stable

* refer to https://github.com/aws/aws-sdk-cpp/blob/main/src/aws-cpp-sdk-core/include/aws/core/client/ClientConfiguration.h
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@SWJTU-ZhangLei
Copy link
Contributor Author

SWJTU-ZhangLei commented Mar 20, 2025

image

@SWJTU-ZhangLei
Copy link
Contributor Author

run buildall

@SWJTU-ZhangLei SWJTU-ZhangLei changed the title [opt](be) Set ClientConfiguration.requestTimeoutMs default vault from 3000ms to 30000ms [opt](be) Set ClientConfiguration.requestTimeoutMs default value from 3000ms to 30000ms Mar 20, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 32902 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 32312b46846ebe3437dc16e13e2595bb85c0d06a, data reload: false

------ Round 1 ----------------------------------
q1	24219	5110	5053	5053
q2	2045	302	174	174
q3	10394	1276	730	730
q4	10226	1042	538	538
q5	7522	2453	2425	2425
q6	200	164	140	140
q7	956	783	639	639
q8	9603	1393	1203	1203
q9	5087	4796	4751	4751
q10	6854	2313	1915	1915
q11	480	274	267	267
q12	366	364	228	228
q13	17762	3697	3049	3049
q14	243	231	209	209
q15	523	486	493	486
q16	622	610	584	584
q17	605	879	367	367
q18	6983	6412	6410	6410
q19	1205	958	593	593
q20	323	333	197	197
q21	2900	2170	1986	1986
q22	1028	1008	958	958
Total cold run time: 110146 ms
Total hot run time: 32902 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5206	5154	5195	5154
q2	245	336	237	237
q3	2165	2679	2291	2291
q4	1439	1824	1383	1383
q5	4233	4206	4458	4206
q6	224	173	123	123
q7	2084	1922	1815	1815
q8	2677	2597	2537	2537
q9	7162	7115	7136	7115
q10	2980	3218	2854	2854
q11	580	516	492	492
q12	715	754	612	612
q13	3572	3895	3303	3303
q14	271	311	283	283
q15	524	488	471	471
q16	659	673	624	624
q17	1171	1617	1377	1377
q18	7831	7440	7421	7421
q19	822	855	981	855
q20	1959	2045	1875	1875
q21	5410	4941	4809	4809
q22	1111	1051	996	996
Total cold run time: 53040 ms
Total hot run time: 50833 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185506 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 32312b46846ebe3437dc16e13e2595bb85c0d06a, data reload: false

query1	1014	476	483	476
query2	6520	1955	1957	1955
query3	6795	216	219	216
query4	26173	23728	23460	23460
query5	4344	655	495	495
query6	307	198	194	194
query7	4599	500	298	298
query8	291	254	252	252
query9	8618	2603	2604	2603
query10	489	319	270	270
query11	15942	15286	14958	14958
query12	179	112	106	106
query13	1674	539	400	400
query14	10801	6724	6343	6343
query15	229	201	173	173
query16	7642	609	494	494
query17	1444	725	578	578
query18	2001	407	309	309
query19	196	189	158	158
query20	123	119	113	113
query21	212	123	107	107
query22	4272	4296	4082	4082
query23	34146	32827	33022	32827
query24	7226	2406	2421	2406
query25	522	440	385	385
query26	1232	268	155	155
query27	2055	491	327	327
query28	3987	2404	2371	2371
query29	733	583	415	415
query30	281	216	192	192
query31	973	823	775	775
query32	74	66	62	62
query33	551	374	292	292
query34	800	847	508	508
query35	792	817	774	774
query36	981	980	886	886
query37	120	98	73	73
query38	4146	4240	4061	4061
query39	1485	1385	1404	1385
query40	202	115	105	105
query41	70	51	48	48
query42	120	102	105	102
query43	496	525	487	487
query44	1373	809	794	794
query45	177	174	164	164
query46	854	1026	628	628
query47	1739	1793	1711	1711
query48	377	412	312	312
query49	799	501	431	431
query50	709	727	401	401
query51	4223	4210	4163	4163
query52	113	115	97	97
query53	233	251	185	185
query54	490	510	397	397
query55	83	80	83	80
query56	281	274	262	262
query57	1117	1123	1110	1110
query58	235	237	242	237
query59	2696	2841	2735	2735
query60	304	285	268	268
query61	131	120	116	116
query62	781	750	676	676
query63	235	188	195	188
query64	4317	1015	660	660
query65	4441	4346	4401	4346
query66	1042	400	300	300
query67	15804	15309	15135	15135
query68	8150	875	489	489
query69	459	300	263	263
query70	1230	1145	1050	1050
query71	476	308	274	274
query72	5705	3573	3776	3573
query73	797	723	346	346
query74	9049	9232	8732	8732
query75	3810	3177	2731	2731
query76	3702	1259	759	759
query77	800	373	285	285
query78	10018	10114	9299	9299
query79	3423	872	576	576
query80	727	533	502	502
query81	508	265	224	224
query82	704	131	103	103
query83	222	175	150	150
query84	288	88	76	76
query85	780	371	309	309
query86	388	309	295	295
query87	4566	4474	4344	4344
query88	3642	2245	2244	2244
query89	403	319	281	281
query90	1872	225	217	217
query91	141	141	109	109
query92	80	62	61	61
query93	1901	1093	574	574
query94	661	424	306	306
query95	367	272	271	271
query96	498	572	277	277
query97	3357	3427	3250	3250
query98	234	209	203	203
query99	1435	1414	1261	1261
Total cold run time: 277804 ms
Total hot run time: 185506 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 32312b46846ebe3437dc16e13e2595bb85c0d06a, data reload: false

query1	0.04	0.03	0.04
query2	0.13	0.10	0.10
query3	0.25	0.19	0.19
query4	1.59	0.18	0.19
query5	0.60	0.59	0.59
query6	1.18	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.58	0.52	0.53
query10	0.57	0.58	0.57
query11	0.15	0.11	0.11
query12	0.15	0.12	0.11
query13	0.61	0.60	0.60
query14	2.80	2.70	2.72
query15	0.92	0.86	0.86
query16	0.39	0.39	0.38
query17	1.02	1.02	1.03
query18	0.22	0.19	0.19
query19	1.88	1.98	1.87
query20	0.02	0.02	0.01
query21	15.35	0.92	0.55
query22	0.78	1.28	0.68
query23	14.82	1.39	0.63
query24	6.74	1.05	1.52
query25	0.52	0.34	0.08
query26	0.52	0.16	0.14
query27	0.05	0.05	0.05
query28	10.25	0.92	0.43
query29	12.56	4.05	3.30
query30	0.25	0.08	0.06
query31	2.82	0.59	0.39
query32	3.23	0.56	0.46
query33	2.95	3.01	2.99
query34	15.73	5.14	4.52
query35	4.59	4.56	4.57
query36	0.68	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.02	0.02	0.02
query40	0.16	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 105.47 s
Total hot run time: 31.5 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 48.80% (13070/26781)
Line Coverage 38.38% (112718/293669)
Region Coverage 37.18% (57298/154130)
Branch Coverage 32.28% (28807/89234)

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 21, 2025
Copy link
Contributor

PR approved by anyone and no changes requested.

@gavinchou gavinchou added the p0_b label Mar 21, 2025
@gavinchou gavinchou merged commit 9308401 into apache:master Mar 22, 2025
34 of 36 checks passed
@gavinchou gavinchou changed the title [opt](be) Set ClientConfiguration.requestTimeoutMs default value from 3000ms to 30000ms [opt](s3client) Set ClientConfiguration.requestTimeoutMs default value from 3000ms to 30000ms Mar 22, 2025
github-actions bot pushed a commit that referenced this pull request Mar 22, 2025
…e from 3000ms to 30000ms (#49315)

* ClientConfiguration.requestTimeoutMs default is 3000ms in
`aws-sdk-cpp`, this should be more than adequate for most services.
However, we meet problem when cloud provider's storage service or
network not stable

* refer to
https://github.com/aws/aws-sdk-cpp/blob/main/src/aws-cpp-sdk-core/include/aws/core/client/ClientConfiguration.h
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.x p0_b reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants