SiteInfoMaxPageIdLookup is failing with:
java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:165) at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:114) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.IllegalArgumentException: Missing property '/query/general/max-page-id' at org.wikimedia.discovery.cirrus.updater.common.model.JsonPathUtils.getRequiredNode(JsonPathUtils.java:13) at org.wikimedia.discovery.cirrus.updater.consumer.sanity.SiteInfoMaxPageIdLookup.parse(SiteInfoMaxPageIdLookup.java:39) at org.wikimedia.discovery.cirrus.updater.common.http.MediaWikiHttpClient.lambda$fetch$0(MediaWikiHttpClient.java:80) at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:247) at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188) at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:162) at org.wikimedia.discovery.cirrus.updater.common.http.MediaWikiHttpClient.fetch(MediaWikiHttpClient.java:73) at org.wikimedia.discovery.cirrus.updater.common.http.MediaWikiHttpClient.load(MediaWikiHttpClient.java:62) at org.wikimedia.discovery.cirrus.updater.consumer.sanity.SiteInfoMaxPageIdLookup.apply(SiteInfoMaxPageIdLookup.java:27) at org.wikimedia.discovery.cirrus.updater.consumer.sanity.SiteInfoMaxPageIdLookup.apply(SiteInfoMaxPageIdLookup.java:15) at org.wikimedia.discovery.cirrus.updater.consumer.sanity.SaneitizeLoop.refreshMaxPageId(SaneitizeLoop.java:255) at org.wikimedia.discovery.cirrus.updater.consumer.sanity.SaneitizeLoop.next(SaneitizeLoop.java:141) at org.wikimedia.discovery.cirrus.updater.consumer.sanity.SanitySourceSplitReader.lambda$fetch$1(SanitySourceSplitReader.java:65) at java.base/java.util.HashMap.forEach(HashMap.java:1337) at org.wikimedia.discovery.cirrus.updater.consumer.sanity.SanitySourceSplitReader.fetch(SanitySourceSplitReader.java:63) at org.wikimedia.discovery.cirrus.updater.consumer.sanity.SanitySourceSplitReader.fetch(SanitySourceSplitReader.java:57) at org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58) at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:162) ... 6 more
AC:
- understand the cause and fix it