Tests fail in mediawiki edit reconstruction in 2 ways:
on 1004:
SubgraphPartitioner
- RUN ABORTED *** java.io.FileNotFoundException: File /tmp/unittest/refinery-source/refinery-job/TestSubgraphPartitioner/2ae8a9df-4da4-4dc7-b9d6-74ae8fbde3be does not exist at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:598) at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:811) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:588) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425) at org.apache.spark.SparkContext$$anonfun$setCheckpointDir$2.apply(SparkContext.scala:2086) at org.apache.spark.SparkContext$$anonfun$setCheckpointDir$2.apply(SparkContext.scala:2082) at scala.Option.map(Option.scala:145) at org.apache.spark.SparkContext.setCheckpointDir(SparkContext.scala:2082) at org.wikimedia.analytics.refinery.job.mediawikihistory.utils.TestSubgraphPartitioner.beforeEach(TestSubgraphPartitioner.scala:38) at org.scalatest.BeforeAndAfterEach$class.beforeEach(BeforeAndAfterEach.scala:154)
On localmachine:
should put max rev ts when no page state match * FAILED *
org.apache.spark.SparkException: Job aborted due to stage failure: Task serialization failed: java.lang.reflect.InvocationTargetException
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
org.apache.spark.io.CompressionCodec$.createCodec(CompressionCodec.scala:72)