Page MenuHomePhabricator

Video transcoding is leaving opening large transactions on s4-master (commons)
Closed, DuplicatePublic

Description

This is basically creating lots of connection issues to the master:

| 19330928 | wikiuser        | 10.64.16.146:36233 | commonswiki        | Sleep       |    3544 |                                                     
| 19334679 | wikiuser        | 10.64.0.197:54489  | commonswiki        | Sleep       |    3397 |                                                     
| 19334749 | wikiuser        | 10.64.0.197:54492  | commonswiki        | Sleep       |    3395 |                                                     
| 19336301 | wikiuser        | 10.64.0.197:54505  | commonswiki        | Sleep       |    3335 |                                                     
| 19339268 | wikiuser        | 10.64.16.146:36313 | commonswiki        | Sleep       |    3227 |                                                     
| 19340851 | wikiuser        | 10.64.16.146:36339 | commonswiki        | Sleep       |    3164 |                                                     
| 19341622 | wikiuser        | 10.64.16.146:36349 | commonswiki        | Sleep       |    3104 |                                                     
| 19342648 | wikiuser        | 10.64.0.197:54558  | commonswiki        | Sleep       |    3093 |                                                     
| 19346824 | wikiuser        | 10.64.16.146:36414 | commonswiki        | Sleep       |    2940 |                                                     
| 19347735 | wikiuser        | 10.64.16.146:36434 | commonswiki        | Sleep       |    2911 |                                                     
| 19348163 | wikiuser        | 10.64.16.146:36453 | commonswiki        | Sleep       |    2896 |                                                     
| 19349153 | wikiuser        | 10.64.16.146:36459 | commonswiki        | Sleep       |    2865 |                                                     
| 19352072 | wikiuser        | 10.64.16.146:36478 | commonswiki        | Sleep       |    2767 |                                                     
| 19365711 | wikiuser        | 10.64.16.146:36628 | commonswiki        | Sleep       |    2279 |                                                     
| 19367711 | wikiuser        | 10.64.0.197:54729  | commonswiki        | Sleep       |    2200 |                                                     
| 19368080 | wikiuser        | 10.64.16.146:36658 | commonswiki        | Sleep       |    2186 |                                                     
| 19369932 | wikiuser        | 10.64.0.197:54738  | commonswiki        | Sleep       |    2126 |                                                     
| 19372351 | wikiuser        | 10.64.0.197:54777  | commonswiki        | Sleep       |    1960 |                                                     
| 19373066 | wikiuser        | 10.64.16.146:36723 | commonswiki        | Sleep       |    2003 |                                                     
| 19374284 | wikiuser        | 10.64.16.146:36926 | commonswiki        | Sleep       |    1955 |

Those 2 ips are of the videoscalers, which have a load of 90.

Not unbreak now because the load (for now) it is not that bad that it has brought the servers down, but it can eventually do (which will force us to kill those idle connections).

I remember a ticket about video transcoding opening long-running transaction to the master (but cannot find the issue), is that still happening?