Page MenuHomePhabricator
Create Task
Maniphest T273867

Kryo memcached transcoder broken in CAS 6.3/6.4
Open, LowPublic
Actions

Description

After upgrading to CAS 6.3 we had received reports that case sessions are being invalidated every 24 hours. This could be related to the recent CAS 6.3.1 updated and requires investigation. These were also reported upstream: https://groups.google.com/u/0/a/apereo.org/g/cas-user/c/MkpgAZZn-Mw

To unbreak session storage we switched the memcached transcoder to the default Java serialiser. It's less space-efficient compared to Kryo, but it works fine.

We can revisit this in a future version and if the transcoder got fixed, revert back to Kryo.

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+36 -36apereo_cas: rename config properties
operations/puppetproduction+1 -1Switch the IDPs to the serial Memcached transcoder
operations/software/cas-overlay-templatemaster+9 -36.4.0-RC2: test to see if issue is still present
Customize query in gerrit

Event Timeline

jbond triaged this task as Medium priority.Feb 4 2021, 11:05 AM
jbond created this task.
Marostegui added a subscriber: Marostegui.Feb 4 2021, 11:07 AM
jbond added a comment.Feb 4 2021, 11:56 AM

The following is from idp-test after updateing case in config

 <Remember me expiration policy is being configured based on hard timeout of [604800] seconds>
2021-02-04 11:50:49,536 DEBUG [org.apereo.cas.ticket.expiration.builder.TicketGrantingTicketExpirationPolicyBuilder] - <Ticket-granting ticket expiration policy is based on hard/idle timeouts of [604800]/[3600] seconds>
2021-02-04 11:50:49,536 DEBUG [org.apereo.cas.ticket.expiration.builder.TicketGrantingTicketExpirationPolicyBuilder] - <Final effective time-to-live of remember-me expiration policy is [604800] seconds>
2021-02-04 11:50:49,540 INFO [org.apereo.inspektr.audit.support.Slf4jLoggingAuditTrailManager] - <Audit trail record BEGIN
jbond added a comment.Feb 4 2021, 12:06 PM

After investigating i notice that even though spring has renamed config properties from camel case (myCoolProperty) to hyphen naming (my-cool-property) it seems old config parameters are silently remapped so this is not the issue. however i notice that when the service is restarted it is unable to retirve the old session from memcached and gives the following stack trace. This measn sessiones are invalidated on every tomcate restart.

likley culprit

Caused by: java.util.concurrent.ExecutionException: com.esotericsoftware.kryo.KryoException: com.esotericsoftware.kryo.KryoException: Invalid ordinal for enum "org.apereo.cas.validation.ValidationResponseType": 16

full

2021-02-04 12:01:33,806 ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <Failed fetching [TGT-1-*****2JZqAmMlII-idp-test1001] >
2021-02-04 12:01:33,808 ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <Exception waiting for value>
java.lang.RuntimeException: Exception waiting for value
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1236) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1256) ~[spymemcached-2.12.3.jar:2.12.3]
        at org.apereo.cas.ticket.registry.MemcachedTicketRegistry.getTicket(MemcachedTicketRegistry.java:96) ~[cas-server-support-memcached-ticket-registry-6.3.1.jar:6.3.1]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:51) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:63) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy135.getTicket(Unknown Source) ~[?:?]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getTicketGrantingTicket(DefaultTicketRegistrySupport.java:45) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getAuthenticationFrom(DefaultTicketRegistrySupport.java:54) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getAuthenticatedPrincipalFrom(DefaultTicketRegistrySupport.java:60) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$FastClassBySpringCGLIB$$885171cb.invoke(<generated>) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367) ~[spring-tx-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118) ~[spring-tx-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$EnhancerBySpringCGLIB$$a738544c.getAuthenticatedPrincipalFrom(<generated>) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.apereo.cas.logging.web.ThreadContextMDCServletFilter.doFilter(ThreadContextMDCServletFilter.java:94) ~[cas-server-core-logging-6.3.1.jar:6.3.1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.inspektr.common.web.ClientInfoThreadLocalFilter.doFilter(ClientInfoThreadLocalFilter.java:66) ~[inspektr-common-1.8.10.GA.jar:1.8.10.GA]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71) ~[log4j-web-2.14.0.jar:2.14.0]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:688) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat9-util-9.0.31.jar:9.0.31]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.util.concurrent.ExecutionException: com.esotericsoftware.kryo.KryoException: com.esotericsoftware.kryo.KryoException: Invalid ordinal for enum "org.apereo.cas.validation.ValidationResponseType": 16
Serialization trace:
clock (org.apereo.cas.ticket.expiration.HardTimeoutExpirationPolicy)
        at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
        at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:97) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3]
        ... 65 more
Caused by: com.esotericsoftware.kryo.KryoException: com.esotericsoftware.kryo.KryoException: Invalid ordinal for enum "org.apereo.cas.validation.ValidationResponseType": 16
Serialization trace:
clock (org.apereo.cas.ticket.expiration.HardTimeoutExpirationPolicy)
        at org.apereo.cas.memcached.kryo.CasKryoTranscoder.decode(CasKryoTranscoder.java:74) ~[cas-server-support-memcached-core-6.3.1.jar:6.3.1]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy254.decode(Unknown Source) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$1.call(TranscodeService.java:63) ~[spymemcached-2.12.3.jar:2.12.3]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.run(TranscodeService.java:110) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:96) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3]
        ... 65 more
Caused by: com.esotericsoftware.kryo.KryoException: Invalid ordinal for enum "org.apereo.cas.validation.ValidationResponseType": 16
Serialization trace:
clock (org.apereo.cas.ticket.expiration.HardTimeoutExpirationPolicy)
        at com.esotericsoftware.kryo.serializers.DefaultSerializers$EnumSerializer.read(DefaultSerializers.java:411) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.DefaultSerializers$EnumSerializer.read(DefaultSerializers.java:378) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:732) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:125) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:124) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813) ~[kryo-5.0.2.jar:?]
        at org.apereo.cas.memcached.kryo.CasKryoTranscoder.decode(CasKryoTranscoder.java:72) ~[cas-server-support-memcached-core-6.3.1.jar:6.3.1]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy254.decode(Unknown Source) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$1.call(TranscodeService.java:63) ~[spymemcached-2.12.3.jar:2.12.3]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.run(TranscodeService.java:110) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:96) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3]
        ... 65 more
2021-02-04 12:01:34,131 ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <Failed fetching [TGT-1-*****2JZqAmMlII-idp-test1001] >
2021-02-04 12:01:34,132 ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <Exception waiting for value>
java.lang.RuntimeException: Exception waiting for value
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1236) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1256) ~[spymemcached-2.12.3.jar:2.12.3]
        at org.apereo.cas.ticket.registry.MemcachedTicketRegistry.getTicket(MemcachedTicketRegistry.java:96) ~[cas-server-support-memcached-ticket-registry-6.3.1.jar:6.3.1]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:51) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:63) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy135.getTicket(Unknown Source) ~[?:?]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getTicketGrantingTicket(DefaultTicketRegistrySupport.java:45) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$FastClassBySpringCGLIB$$885171cb.invoke(<generated>) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367) ~[spring-tx-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118) ~[spring-tx-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$EnhancerBySpringCGLIB$$a738544c.getTicketGrantingTicket(<generated>) ~[cas-server-core-tickets-api-6.3.1.jar:6.3.1]
        at org.apereo.cas.web.flow.login.InitialFlowSetupAction.configureWebflowForTicketGrantingTicket(InitialFlowSetupAction.java:90) ~[cas-server-support-actions-6.3.1.jar:6.3.1]
        at org.apereo.cas.web.flow.login.InitialFlowSetupAction.doExecute(InitialFlowSetupAction.java:73) ~[cas-server-support-actions-6.3.1.jar:6.3.1]
        at org.springframework.webflow.action.AbstractAction.execute(AbstractAction.java:188) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy273.execute(Unknown Source) ~[?:?]
        at org.springframework.webflow.execution.ActionExecutor.execute(ActionExecutor.java:51) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.action.EvaluateAction.doExecute(EvaluateAction.java:77) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.action.AbstractAction.execute(AbstractAction.java:188) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.execution.ActionExecutor.execute(ActionExecutor.java:51) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.engine.ActionList.execute(ActionList.java:154) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.engine.Flow.start(Flow.java:526) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.engine.impl.FlowExecutionImpl.start(FlowExecutionImpl.java:368) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.engine.impl.FlowExecutionImpl.start(FlowExecutionImpl.java:223) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.executor.FlowExecutorImpl.launchExecution(FlowExecutorImpl.java:139) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy296.launchExecution(Unknown Source) ~[?:?]
        at org.springframework.webflow.mvc.servlet.FlowHandlerAdapter.handle(FlowHandlerAdapter.java:264) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) ~[spring-webmvc-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) ~[spring-webmvc-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) ~[spring-webmvc-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) ~[tomcat9-servlet-api-9.0.31.jar:?]
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat9-servlet-api-9.0.31.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat9-websocket-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.web.support.AuthenticationCredentialsThreadLocalBinderClearingFilter.doFilter(AuthenticationCredentialsThreadLocalBinderClearingFilter.java:28) ~[cas-server-core-web-api-6.3.1.jar:6.3.1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.web.support.filters.RequestParameterPolicyEnforcementFilter.doFilter(RequestParameterPolicyEnforcementFilter.java:401) ~[cas-server-core-web-api-6.3.1.jar:6.3.1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.web.support.filters.ResponseHeadersEnforcementFilter.doFilter(ResponseHeadersEnforcementFilter.java:200) ~[cas-server-core-web-api-6.3.1.jar:6.3.1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.web.support.filters.AddResponseHeadersFilter.doFilter(AddResponseHeadersFilter.java:64) ~[cas-server-core-web-api-6.3.1.jar:6.3.1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:204) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.security.web.debug.DebugFilter.invokeWithWrappedRequest(DebugFilter.java:90) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.security.web.debug.DebugFilter.doFilter(DebugFilter.java:78) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.security.web.debug.DebugFilter.doFilter(DebugFilter.java:67) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93) ~[spring-boot-actuator-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.logging.web.ThreadContextMDCServletFilter.doFilter(ThreadContextMDCServletFilter.java:99) ~[cas-server-core-logging-6.3.1.jar:6.3.1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.inspektr.common.web.ClientInfoThreadLocalFilter.doFilter(ClientInfoThreadLocalFilter.java:66) ~[inspektr-common-1.8.10.GA.jar:1.8.10.GA]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71) ~[log4j-web-2.14.0.jar:2.14.0]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:688) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat9-util-9.0.31.jar:9.0.31]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.util.concurrent.ExecutionException: com.esotericsoftware.kryo.KryoException: com.esotericsoftware.kryo.KryoException: Invalid ordinal for enum "org.apereo.cas.validation.ValidationResponseType": 16
Serialization trace:
clock (org.apereo.cas.ticket.expiration.HardTimeoutExpirationPolicy)
        at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
        at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:97) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3]
        ... 143 more
Caused by: com.esotericsoftware.kryo.KryoException: com.esotericsoftware.kryo.KryoException: Invalid ordinal for enum "org.apereo.cas.validation.ValidationResponseType": 16
Serialization trace:
clock (org.apereo.cas.ticket.expiration.HardTimeoutExpirationPolicy)
        at org.apereo.cas.memcached.kryo.CasKryoTranscoder.decode(CasKryoTranscoder.java:74) ~[cas-server-support-memcached-core-6.3.1.jar:6.3.1]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy254.decode(Unknown Source) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$1.call(TranscodeService.java:63) ~[spymemcached-2.12.3.jar:2.12.3]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.run(TranscodeService.java:110) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:96) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3]
        ... 143 more
Caused by: com.esotericsoftware.kryo.KryoException: Invalid ordinal for enum "org.apereo.cas.validation.ValidationResponseType": 16
Serialization trace:
clock (org.apereo.cas.ticket.expiration.HardTimeoutExpirationPolicy)
        at com.esotericsoftware.kryo.serializers.DefaultSerializers$EnumSerializer.read(DefaultSerializers.java:411) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.DefaultSerializers$EnumSerializer.read(DefaultSerializers.java:378) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:732) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:125) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:124) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813) ~[kryo-5.0.2.jar:?]
        at org.apereo.cas.memcached.kryo.CasKryoTranscoder.decode(CasKryoTranscoder.java:72) ~[cas-server-support-memcached-core-6.3.1.jar:6.3.1]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy254.decode(Unknown Source) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$1.call(TranscodeService.java:63) ~[spymemcached-2.12.3.jar:2.12.3]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.run(TranscodeService.java:110) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:96) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3]
        ... 143 more
gerritbot added a comment.Feb 4 2021, 12:16 PM

Change 661713 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] apereo_cas: rename config properties

https://gerrit.wikimedia.org/r/661713

gerritbot added a project: Patch-For-Review.Feb 4 2021, 12:16 PM
Stashbot added a comment.Feb 4 2021, 1:10 PM

Mentioned in SAL (#wikimedia-operations) [2021-02-04T13:10:31Z] <jbond42> upload cas_6.2.7 to downgrade cas T273867

gerritbot added a comment.Feb 4 2021, 1:29 PM

Change 661721 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/software/cas-overlay-template@master] 6.4.0-RC1: test to see if issue is still present

https://gerrit.wikimedia.org/r/661721

jbond added a comment.Feb 4 2021, 1:37 PM

This appears to be a regression related to the KRYO serialisation. As far as i can tell it only has issues after a restart. Early musing wonder if there is some nonce initiated on startup?. For now i have downgraded cas to 6.2.7 which appears to work and will debug on idp-test.

jbond added a comment.Feb 4 2021, 1:48 PM

look srelated https://groups.google.com/g/jasig-cas-user/c/v2VTr1y_X8M/m/_gieSp0lDAAJ

jbond added a comment.Feb 4 2021, 2:06 PM

Simlar issue with 6.4.0-RC1

java.lang.ClassCastException: class org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult cannot be cast to class org.apereo.cas.ticket.Ticket (org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult and org.apereo.cas.ticket.Ticket are in unnamed module of loader org.apache.catalina.loader.ParallelWebappClassLoader @686449f9)
 ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <class org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult cannot be cast to class org.apereo.cas.ticket.Ticket (org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult and org.apereo.cas.ticket.Ticket are in unnamed module of loader org.apache.catalina.loader.ParallelWebappClassLoader @686449f9)>
ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <Failed fetching [TGT-2-*****QFIOFbxGko-idp-test1001] >
2021-02-04 14:03:52,617 ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <class org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult cannot be cast to class org.apereo.cas.ticket.Ticket (org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult and org.apereo.cas.ticket.Ticket are in unnamed module of loader org.apache.catalina.loader.ParallelWebappClassLoader @686449f9)>
java.lang.ClassCastException: class org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult cannot be cast to class org.apereo.cas.ticket.Ticket (org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult and org.apereo.cas.ticket.Ticket are in unnamed module of loader org.apache.catalina.loader.ParallelWebappClassLoader @686449f9)
        at org.apereo.cas.ticket.registry.MemcachedTicketRegistry.getTicket(MemcachedTicketRegistry.java:96) ~[cas-server-support-memcached-ticket-registry-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:50) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:62) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.3.3.jar:5.3.3]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:490) ~[spring-cloud-context-3.0.1.jar:3.0.1]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.3.jar:5.3.3]
        at com.sun.proxy.$Proxy138.getTicket(Unknown Source) ~[?:?]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getTicketGrantingTicket(DefaultTicketRegistrySupport.java:45) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getAuthenticationFrom(DefaultTicketRegistrySupport.java:54) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getAuthenticatedPrincipalFrom(DefaultTicketRegistrySupport.java:60) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$FastClassBySpringCGLIB$$885171cb.invoke(<generated>) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123) ~[spring-tx-5.3.3.jar:5.3.3]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[spring-tx-5.3.3.jar:5.3.3]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) ~[spring-tx-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$EnhancerBySpringCGLIB$$c8a92130.getAuthenticatedPrincipalFrom(<generated>) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.logging.web.ThreadContextMDCServletFilter.doFilter(ThreadContextMDCServletFilter.java:94) ~[cas-server-core-logging-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.inspektr.common.web.ClientInfoThreadLocalFilter.doFilter(ClientInfoThreadLocalFilter.java:66) ~[inspektr-common-1.8.14.GA.jar:1.8.14.GA]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.3.jar:5.3.3]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.3.3.jar:5.3.3]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.3.jar:5.3.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71) ~[log4j-web-2.14.0.jar:2.14.0]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:688) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat9-util-9.0.31.jar:9.0.31]
        at java.lang.Thread.run(Thread.java:834) [?:?]
2021-02-04 14:03:52,831 ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <Failed fetching [TGT-2-*****QFIOFbxGko-idp-test1001] >
2021-02-04 14:03:52,831 ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <class org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult cannot be cast to class org.apereo.cas.ticket.Ticket (org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult and org.apereo.cas.ticket.Ticket are in unnamed module of loader org.apache.catalina.loader.ParallelWebappClassLoader @686449f9)>
java.lang.ClassCastException: class org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult cannot be cast to class org.apereo.cas.ticket.Ticket (org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult and org.apereo.cas.ticket.Ticket are in unnamed module of loader org.apache.catalina.loader.ParallelWebappClassLoader @686449f9)
        at org.apereo.cas.ticket.registry.MemcachedTicketRegistry.getTicket(MemcachedTicketRegistry.java:96) ~[cas-server-support-memcached-ticket-registry-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:50) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:62) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.3.3.jar:5.3.3]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:490) ~[spring-cloud-context-3.0.1.jar:3.0.1]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.3.jar:5.3.3]
        at com.sun.proxy.$Proxy138.getTicket(Unknown Source) ~[?:?]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getTicketGrantingTicket(DefaultTicketRegistrySupport.java:45) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$FastClassBySpringCGLIB$$885171cb.invoke(<generated>) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123) ~[spring-tx-5.3.3.jar:5.3.3]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[spring-tx-5.3.3.jar:5.3.3]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) ~[spring-tx-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$EnhancerBySpringCGLIB$$c8a92130.getTicketGrantingTicket(<generated>) ~[cas-server-core-tickets-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.web.flow.login.InitialFlowSetupAction.configureWebflowForTicketGrantingTicket(InitialFlowSetupAction.java:90) ~[cas-server-support-actions-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apereo.cas.web.flow.login.InitialFlowSetupAction.doExecute(InitialFlowSetupAction.java:73) ~[cas-server-support-actions-6.4.0-RC1.jar:6.4.0-RC1]
        at org.springframework.webflow.action.AbstractAction.execute(AbstractAction.java:188) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.3.3.jar:5.3.3]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:490) ~[spring-cloud-context-3.0.1.jar:3.0.1]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.3.jar:5.3.3]
        at com.sun.proxy.$Proxy280.execute(Unknown Source) ~[?:?]
        at org.springframework.webflow.execution.ActionExecutor.execute(ActionExecutor.java:51) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.action.EvaluateAction.doExecute(EvaluateAction.java:77) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.action.AbstractAction.execute(AbstractAction.java:188) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.execution.ActionExecutor.execute(ActionExecutor.java:51) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.engine.ActionList.execute(ActionList.java:154) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.engine.Flow.start(Flow.java:526) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.engine.impl.FlowExecutionImpl.start(FlowExecutionImpl.java:368) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.engine.impl.FlowExecutionImpl.start(FlowExecutionImpl.java:223) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.webflow.executor.FlowExecutorImpl.launchExecution(FlowExecutorImpl.java:139) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.3.3.jar:5.3.3]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:490) ~[spring-cloud-context-3.0.1.jar:3.0.1]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.3.jar:5.3.3]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.3.jar:5.3.3]
        at com.sun.proxy.$Proxy303.launchExecution(Unknown Source) ~[?:?]
        at org.springframework.webflow.mvc.servlet.FlowHandlerAdapter.handle(FlowHandlerAdapter.java:264) ~[spring-webflow-2.5.1.RELEASE.jar:2.5.1.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1060) ~[spring-webmvc-5.3.3.jar:5.3.3]
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:962) ~[spring-webmvc-5.3.3.jar:5.3.3]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.3.3.jar:5.3.3]
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) ~[spring-webmvc-5.3.3.jar:5.3.3]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) ~[tomcat9-servlet-api-9.0.31.jar:?]
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.3.3.jar:5.3.3]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat9-servlet-api-9.0.31.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat9-websocket-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.web.support.AuthenticationCredentialsThreadLocalBinderClearingFilter.doFilter(AuthenticationCredentialsThreadLocalBinderClearingFilter.java:28) ~[cas-server-core-web-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.web.support.filters.RequestParameterPolicyEnforcementFilter.doFilter(RequestParameterPolicyEnforcementFilter.java:401) ~[cas-server-core-web-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.web.support.filters.ResponseHeadersEnforcementFilter.doFilter(ResponseHeadersEnforcementFilter.java:200) ~[cas-server-core-web-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.web.support.filters.AddResponseHeadersFilter.doFilter(AddResponseHeadersFilter.java:64) ~[cas-server-core-web-api-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) ~[spring-web-5.3.3.jar:5.3.3]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.3.jar:5.3.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:204) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.security.web.debug.DebugFilter.invokeWithWrappedRequest(DebugFilter.java:90) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.security.web.debug.DebugFilter.doFilter(DebugFilter.java:78) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.security.web.debug.DebugFilter.doFilter(DebugFilter.java:67) ~[spring-security-web-5.4.2.jar:5.4.2]
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) ~[spring-web-5.3.3.jar:5.3.3]
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) ~[spring-web-5.3.3.jar:5.3.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-5.3.3.jar:5.3.3]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.3.jar:5.3.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-5.3.3.jar:5.3.3]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.3.jar:5.3.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93) ~[spring-boot-actuator-2.4.2.jar:2.4.2]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.3.jar:5.3.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.cas.logging.web.ThreadContextMDCServletFilter.doFilter(ThreadContextMDCServletFilter.java:99) ~[cas-server-core-logging-6.4.0-RC1.jar:6.4.0-RC1]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.inspektr.common.web.ClientInfoThreadLocalFilter.doFilter(ClientInfoThreadLocalFilter.java:66) ~[inspektr-common-1.8.14.GA.jar:1.8.14.GA]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.3.jar:5.3.3]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) ~[spring-boot-2.4.2.jar:2.4.2]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.3.3.jar:5.3.3]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.3.jar:5.3.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71) ~[log4j-web-2.14.0.jar:2.14.0]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:688) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat9-util-9.0.31.jar:9.0.31]
        at java.lang.Thread.run(Thread.java:834) [?:?]
jbond added a comment.Feb 4 2021, 3:58 PM

Testing different transcoders shows this is only an issue with KYRO:

cas eversiontranscoderstatus
6.3.1KRYOfail
6.4.0-RC1KRYOfail
6.4.0-RC1SERIALworks
6.3.1SERIALworks
6.4.0-RC1WHALINworks
6.3.1WHALINworks
6.4.0-RC1WHALINV1works
6.3.1WHALINV1works
jbond updated the task description. (Show Details)Feb 4 2021, 4:00 PM
jbond added a comment.EditedFeb 4 2021, 4:29 PM

git diff v6.2.7..v6.3.0-RC3 support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo
(github kept crashing)

diff --git a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/CasKryoPool.java b/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/CasKryoPool.java
index aab104bb449..eb4f8a5ce75 100644
--- a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/CasKryoPool.java
+++ b/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/CasKryoPool.java
@@ -16,17 +16,12 @@ import java.util.Collection;
  */
 public class CasKryoPool implements KryoPool {
 
-
     private final KryoPool kryoPoolRef;
 
     public CasKryoPool() {
         this(new ArrayList<>(0), true, true, false, false);
     }
 
-    public CasKryoPool(final Collection<Class> classesToRegister) {
-        this(classesToRegister, true, true, false, false);
-    }
-
     public CasKryoPool(final Collection<Class> classesToRegister,
                        final boolean warnUnregisteredClasses,
                        final boolean registrationRequired,
diff --git a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/CloseableKryoFactory.java b/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/CloseableKryoFactory.java
index 1f4db86536d..9f3da6a93a8 100644
--- a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/CloseableKryoFactory.java
+++ b/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/CloseableKryoFactory.java
@@ -1,98 +1,11 @@
 package org.apereo.cas.memcached.kryo;
 
-import org.apereo.cas.DefaultMessageDescriptor;
-import org.apereo.cas.authentication.AttributeMergingStrategy;
-import org.apereo.cas.authentication.DefaultAuthentication;
-import org.apereo.cas.authentication.DefaultAuthenticationHandlerExecutionResult;
-import org.apereo.cas.authentication.PreventedException;
-import org.apereo.cas.authentication.PrincipalException;
-import org.apereo.cas.authentication.credential.BasicIdentifiableCredential;
-import org.apereo.cas.authentication.credential.HttpBasedServiceCredential;
-import org.apereo.cas.authentication.credential.OneTimePasswordCredential;
-import org.apereo.cas.authentication.credential.RememberMeUsernamePasswordCredential;
-import org.apereo.cas.authentication.credential.UsernamePasswordCredential;
-import org.apereo.cas.authentication.exceptions.AccountDisabledException;
-import org.apereo.cas.authentication.exceptions.InvalidLoginLocationException;
-import org.apereo.cas.authentication.exceptions.InvalidLoginTimeException;
-import org.apereo.cas.authentication.exceptions.MixedPrincipalException;
-import org.apereo.cas.authentication.metadata.BasicCredentialMetaData;
-import org.apereo.cas.authentication.principal.DefaultPrincipalAttributesRepository;
-import org.apereo.cas.authentication.principal.ShibbolethCompatiblePersistentIdGenerator;
-import org.apereo.cas.authentication.principal.SimplePrincipal;
-import org.apereo.cas.authentication.principal.SimpleWebApplicationServiceImpl;
-import org.apereo.cas.authentication.principal.cache.CachingPrincipalAttributesRepository;
-import org.apereo.cas.authentication.support.password.PasswordExpiringWarningMessageDescriptor;
 import org.apereo.cas.memcached.kryo.serial.ImmutableNativeJavaListSerializer;
+import org.apereo.cas.memcached.kryo.serial.ImmutableNativeJavaMapSerializer;
 import org.apereo.cas.memcached.kryo.serial.ImmutableNativeJavaSetSerializer;
-import org.apereo.cas.memcached.kryo.serial.RegisteredServiceSerializer;
-import org.apereo.cas.memcached.kryo.serial.SimpleWebApplicationServiceSerializer;
 import org.apereo.cas.memcached.kryo.serial.ThrowableSerializer;
 import org.apereo.cas.memcached.kryo.serial.URLSerializer;
 import org.apereo.cas.memcached.kryo.serial.ZonedDateTimeSerializer;
-import org.apereo.cas.services.AllAuthenticationHandlersRegisteredServiceAuthenticationPolicyCriteria;
-import org.apereo.cas.services.AllowedAuthenticationHandlersRegisteredServiceAuthenticationPolicyCriteria;
-import org.apereo.cas.services.AnonymousRegisteredServiceUsernameAttributeProvider;
-import org.apereo.cas.services.AnyAuthenticationHandlerRegisteredServiceAuthenticationPolicyCriteria;
-import org.apereo.cas.services.ChainingAttributeReleasePolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceAcceptableUsagePolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceAccessStrategy;
-import org.apereo.cas.services.DefaultRegisteredServiceAuthenticationPolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceContact;
-import org.apereo.cas.services.DefaultRegisteredServiceDelegatedAuthenticationPolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceExpirationPolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceMultifactorPolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceProperty;
-import org.apereo.cas.services.DefaultRegisteredServiceProxyTicketExpirationPolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceServiceTicketExpirationPolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceUsernameProvider;
-import org.apereo.cas.services.DenyAllAttributeReleasePolicy;
-import org.apereo.cas.services.GroovyRegisteredServiceAccessStrategy;
-import org.apereo.cas.services.GroovyRegisteredServiceAuthenticationPolicyCriteria;
-import org.apereo.cas.services.GroovyRegisteredServiceMultifactorPolicy;
-import org.apereo.cas.services.GroovyRegisteredServiceUsernameProvider;
-import org.apereo.cas.services.GroovyScriptAttributeReleasePolicy;
-import org.apereo.cas.services.NotPreventedRegisteredServiceAuthenticationPolicyCriteria;
-import org.apereo.cas.services.PrincipalAttributeRegisteredServiceUsernameProvider;
-import org.apereo.cas.services.RefuseRegisteredServiceProxyPolicy;
-import org.apereo.cas.services.RegexMatchingRegisteredServiceProxyPolicy;
-import org.apereo.cas.services.RegexRegisteredService;
-import org.apereo.cas.services.RegisteredServiceLogoutType;
-import org.apereo.cas.services.RegisteredServiceMultifactorPolicyFailureModes;
-import org.apereo.cas.services.RegisteredServicePublicKeyImpl;
-import org.apereo.cas.services.RemoteEndpointServiceAccessStrategy;
-import org.apereo.cas.services.RestfulRegisteredServiceAuthenticationPolicyCriteria;
-import org.apereo.cas.services.ReturnAllAttributeReleasePolicy;
-import org.apereo.cas.services.ReturnAllowedAttributeReleasePolicy;
-import org.apereo.cas.services.ReturnMappedAttributeReleasePolicy;
-import org.apereo.cas.services.ReturnRestfulAttributeReleasePolicy;
-import org.apereo.cas.services.ScriptedRegisteredServiceAttributeReleasePolicy;
-import org.apereo.cas.services.ScriptedRegisteredServiceUsernameProvider;
-import org.apereo.cas.services.TimeBasedRegisteredServiceAccessStrategy;
-import org.apereo.cas.services.UnauthorizedServiceException;
-import org.apereo.cas.services.UnauthorizedServiceForPrincipalException;
-import org.apereo.cas.services.UnauthorizedSsoServiceException;
-import org.apereo.cas.services.consent.DefaultRegisteredServiceConsentPolicy;
-import org.apereo.cas.services.support.RegisteredServiceChainingAttributeFilter;
-import org.apereo.cas.services.support.RegisteredServiceMappedRegexAttributeFilter;
-import org.apereo.cas.services.support.RegisteredServiceRegexAttributeFilter;
-import org.apereo.cas.services.support.RegisteredServiceScriptedAttributeFilter;
-import org.apereo.cas.ticket.ProxyGrantingTicketImpl;
-import org.apereo.cas.ticket.ProxyTicketImpl;
-import org.apereo.cas.ticket.ServiceTicketImpl;
-import org.apereo.cas.ticket.TicketGrantingTicketImpl;
-import org.apereo.cas.ticket.TransientSessionTicketImpl;
-import org.apereo.cas.ticket.expiration.AlwaysExpiresExpirationPolicy;
-import org.apereo.cas.ticket.expiration.BaseDelegatingExpirationPolicy;
-import org.apereo.cas.ticket.expiration.HardTimeoutExpirationPolicy;
-import org.apereo.cas.ticket.expiration.MultiTimeUseOrTimeoutExpirationPolicy;
-import org.apereo.cas.ticket.expiration.NeverExpiresExpirationPolicy;
-import org.apereo.cas.ticket.expiration.RememberMeDelegatingExpirationPolicy;
-import org.apereo.cas.ticket.expiration.ThrottledUseAndTimeoutExpirationPolicy;
-import org.apereo.cas.ticket.expiration.TicketGrantingTicketExpirationPolicy;
-import org.apereo.cas.ticket.expiration.TimeoutExpirationPolicy;
-import org.apereo.cas.ticket.registry.EncodedTicket;
-import org.apereo.cas.util.crypto.PublicKeyFactoryBean;
-import org.apereo.cas.validation.ValidationResponseType;
 
 import com.esotericsoftware.kryo.Kryo;
 import com.esotericsoftware.kryo.pool.KryoFactory;
@@ -124,8 +37,6 @@ import org.joda.time.DateTime;
 import org.joda.time.LocalDateTime;
 import org.objenesis.strategy.StdInstantiatorStrategy;
 
-import javax.security.auth.login.AccountExpiredException;
-import javax.security.auth.login.AccountLockedException;
 import javax.security.auth.login.AccountNotFoundException;
 import java.net.URI;
 import java.net.URL;
@@ -195,23 +106,8 @@ public class CloseableKryoFactory implements KryoFactory {
         LOGGER.debug("replaceObjectsByReferences: [{}]", this.replaceObjectsByReferences);
         LOGGER.debug("registrationRequired: [{}]", this.registrationRequired);
 
-        registerCasAuthenticationWithKryo(kryo);
-        registerExpirationPoliciesWithKryo(kryo);
-        registerCasTicketsWithKryo(kryo);
         registerNativeJdkComponentsWithKryo(kryo);
-        registerCasServicesWithKryo(kryo);
-        registerCasServicesAttributeFiltersWithKryo(kryo);
-        registerCasServicesUsernameAttributeProvidersWithKryo(kryo);
-        registerCasServicesAccessStrategyWithKryo(kryo);
         registerImmutableOrEmptyCollectionsWithKryo(kryo);
-        registerCasServicesProxyPolicyWithKryo(kryo);
-        registerExceptionsWithKryo(kryo);
-        registerMessageDescriptorsWithKryo(kryo);
-        registerCasServicesPrincipalAttributeRepositoryWithKryo(kryo);
-        registerCasServicesMultifactorPolicyWithKryo(kryo);
-        registerCasServicesConsentPolicyWithKryo(kryo);
-        registerCasServicesAttributeReleasePolicyWithKryo(kryo);
-        registerCasServicesAuthenticationPolicy(kryo);
 
         classesToRegister.forEach(c -> {
             LOGGER.trace("Registering serializable class [{}] with Kryo", c.getName());
@@ -221,24 +117,27 @@ public class CloseableKryoFactory implements KryoFactory {
     }
 
     private static void registerImmutableOrEmptyCollectionsWithKryo(final Kryo kryo) {
-        LOGGER.debug("Registering immutable/empty collections with Kryo");
+        LOGGER.trace("Registering immutable/empty collections with Kryo");
 
         UnmodifiableCollectionsSerializer.registerSerializers(kryo);
 
         ImmutableListSerializer.registerSerializers(kryo);
         kryo.register(List.of().getClass(), new ImmutableNativeJavaListSerializer());
+        kryo.register(List.class, new ImmutableNativeJavaListSerializer());
         kryo.register(List.of("1", "2").getClass(), new ImmutableNativeJavaListSerializer());
         kryo.register(List.of("1", "2", "3", "4").getClass(), new ImmutableNativeJavaListSerializer());
 
         ImmutableSetSerializer.registerSerializers(kryo);
         kryo.register(Set.of().getClass(), new ImmutableNativeJavaSetSerializer());
+        kryo.register(Set.class, new ImmutableNativeJavaSetSerializer());
         kryo.register(Set.of("1", "2").getClass(), new ImmutableNativeJavaSetSerializer());
         kryo.register(Set.of("1", "2", "3", "4").getClass(), new ImmutableNativeJavaSetSerializer());
 
         ImmutableMapSerializer.registerSerializers(kryo);
-        kryo.register(Map.of().getClass(), new ImmutableMapSerializer());
-        kryo.register(Map.of("1", "2").getClass(), new ImmutableMapSerializer());
-        kryo.register(Map.of("1", "2", "3", "4", "5", "6").getClass(), new ImmutableMapSerializer());
+        kryo.register(Map.of().getClass(), new ImmutableNativeJavaMapSerializer());
+        kryo.register(Map.class, new ImmutableNativeJavaMapSerializer());
+        kryo.register(Map.of("1", "2").getClass(), new ImmutableNativeJavaMapSerializer());
+        kryo.register(Map.of("1", "2", "3", "4", "5", "6").getClass(), new ImmutableNativeJavaMapSerializer());
 
         ImmutableMultimapSerializer.registerSerializers(kryo);
 
@@ -263,117 +162,10 @@ public class CloseableKryoFactory implements KryoFactory {
         kryo.register(list.getClass(), new ArraysAsListSerializer());
     }
 
-    private static void registerCasServicesWithKryo(final Kryo kryo) {
-        kryo.register(RegexRegisteredService.class, new RegisteredServiceSerializer());
-        kryo.register(RegisteredServiceLogoutType.class);
-        kryo.register(RegisteredServicePublicKeyImpl.class);
-        kryo.register(DefaultRegisteredServiceContact.class);
-        kryo.register(DefaultRegisteredServiceProperty.class);
-        kryo.register(DefaultRegisteredServiceDelegatedAuthenticationPolicy.class);
-        kryo.register(DefaultRegisteredServiceExpirationPolicy.class);
-        kryo.register(DefaultRegisteredServiceServiceTicketExpirationPolicy.class);
-        kryo.register(DefaultRegisteredServiceProxyTicketExpirationPolicy.class);
-        kryo.register(DefaultRegisteredServiceDelegatedAuthenticationPolicy.class);
-        kryo.register(DefaultRegisteredServiceAcceptableUsagePolicy.class);
-        kryo.register(DefaultRegisteredServiceAuthenticationPolicy.class);
-        kryo.register(ShibbolethCompatiblePersistentIdGenerator.class);
-    }
-
-    private static void registerCasServicesProxyPolicyWithKryo(final Kryo kryo) {
-        kryo.register(RegexMatchingRegisteredServiceProxyPolicy.class);
-        kryo.register(RefuseRegisteredServiceProxyPolicy.class);
-    }
-
-    private static void registerCasServicesAccessStrategyWithKryo(final Kryo kryo) {
-        kryo.register(DefaultRegisteredServiceAccessStrategy.class);
-        kryo.register(GroovyRegisteredServiceAccessStrategy.class);
-        kryo.register(RemoteEndpointServiceAccessStrategy.class);
-        kryo.register(TimeBasedRegisteredServiceAccessStrategy.class);
-    }
-
-    private static void registerCasServicesUsernameAttributeProvidersWithKryo(final Kryo kryo) {
-        kryo.register(PrincipalAttributeRegisteredServiceUsernameProvider.class);
-        kryo.register(AnonymousRegisteredServiceUsernameAttributeProvider.class);
-        kryo.register(GroovyRegisteredServiceUsernameProvider.class);
-        kryo.register(DefaultRegisteredServiceUsernameProvider.class);
-        kryo.register(ScriptedRegisteredServiceUsernameProvider.class);
-    }
-
-    private static void registerCasServicesAttributeFiltersWithKryo(final Kryo kryo) {
-        kryo.register(RegisteredServiceRegexAttributeFilter.class);
-        kryo.register(RegisteredServiceChainingAttributeFilter.class);
-        kryo.register(RegisteredServiceMappedRegexAttributeFilter.class);
-        kryo.register(RegisteredServiceScriptedAttributeFilter.class);
-    }
-
-    private static void registerCasAuthenticationWithKryo(final Kryo kryo) {
-        kryo.register(SimpleWebApplicationServiceImpl.class, new SimpleWebApplicationServiceSerializer());
-        kryo.register(BasicCredentialMetaData.class);
-        kryo.register(BasicIdentifiableCredential.class);
-        kryo.register(DefaultAuthenticationHandlerExecutionResult.class);
-        kryo.register(DefaultAuthentication.class);
-        kryo.register(UsernamePasswordCredential.class);
-        kryo.register(RememberMeUsernamePasswordCredential.class);
-        kryo.register(SimplePrincipal.class);
-        kryo.register(HttpBasedServiceCredential.class);
-        kryo.register(OneTimePasswordCredential.class);
-        kryo.register(PublicKeyFactoryBean.class);
-        kryo.register(ValidationResponseType.class);
-    }
-
-    private static void registerCasServicesAttributeReleasePolicyWithKryo(final Kryo kryo) {
-        kryo.register(ChainingAttributeReleasePolicy.class);
-        kryo.register(DenyAllAttributeReleasePolicy.class);
-        kryo.register(ReturnAllowedAttributeReleasePolicy.class);
-        kryo.register(ReturnAllAttributeReleasePolicy.class);
-        kryo.register(ReturnMappedAttributeReleasePolicy.class);
-        kryo.register(GroovyScriptAttributeReleasePolicy.class);
-        kryo.register(ScriptedRegisteredServiceAttributeReleasePolicy.class);
-        kryo.register(ReturnRestfulAttributeReleasePolicy.class);
-    }
-
-    private static void registerCasServicesConsentPolicyWithKryo(final Kryo kryo) {
-        kryo.register(DefaultRegisteredServiceConsentPolicy.class);
-    }
-
-    private static void registerCasServicesMultifactorPolicyWithKryo(final Kryo kryo) {
-        kryo.register(DefaultRegisteredServiceMultifactorPolicy.class);
-        kryo.register(GroovyRegisteredServiceMultifactorPolicy.class);
-        kryo.register(RegisteredServiceMultifactorPolicyFailureModes.class);
-    }
-
-    private static void registerCasServicesPrincipalAttributeRepositoryWithKryo(final Kryo kryo) {
-        kryo.register(CachingPrincipalAttributesRepository.class);
-        kryo.register(DefaultPrincipalAttributesRepository.class);
-        kryo.register(AttributeMergingStrategy.class);
-    }
-
-    private static void registerExceptionsWithKryo(final Kryo kryo) {
+    private static void registerNativeJdkComponentsWithKryo(final Kryo kryo) {
         kryo.register(GeneralSecurityException.class, new ThrowableSerializer());
-        kryo.register(PreventedException.class);
         kryo.register(AccountNotFoundException.class, new ThrowableSerializer());
-        kryo.register(AccountDisabledException.class);
-        kryo.register(AccountExpiredException.class);
-        kryo.register(AccountLockedException.class);
-        kryo.register(InvalidLoginLocationException.class);
-        kryo.register(InvalidLoginTimeException.class);
-        kryo.register(PrincipalException.class);
-        kryo.register(MixedPrincipalException.class);
-        kryo.register(UnauthorizedServiceException.class);
-        kryo.register(UnauthorizedServiceForPrincipalException.class);
-        kryo.register(UnauthorizedSsoServiceException.class);
-    }
-
-    private static void registerCasTicketsWithKryo(final Kryo kryo) {
-        kryo.register(TicketGrantingTicketImpl.class);
-        kryo.register(ServiceTicketImpl.class);
-        kryo.register(ProxyGrantingTicketImpl.class);
-        kryo.register(ProxyTicketImpl.class);
-        kryo.register(EncodedTicket.class);
-        kryo.register(TransientSessionTicketImpl.class);
-    }
 
-    private static void registerNativeJdkComponentsWithKryo(final Kryo kryo) {
         kryo.register(Class.class, new DefaultSerializers.ClassSerializer());
         kryo.register(ArrayList.class);
         kryo.register(LinkedList.class);
@@ -414,32 +206,4 @@ public class CloseableKryoFactory implements KryoFactory {
         kryo.register(ZoneOffset.class);
         kryo.register(EnumSet.class, new EnumSetSerializer());
     }
-
-    private static void registerExpirationPoliciesWithKryo(final Kryo kryo) {
-        kryo.register(MultiTimeUseOrTimeoutExpirationPolicy.class);
-        kryo.register(MultiTimeUseOrTimeoutExpirationPolicy.ServiceTicketExpirationPolicy.class);
-        kryo.register(MultiTimeUseOrTimeoutExpirationPolicy.ProxyTicketExpirationPolicy.class);
-        kryo.register(NeverExpiresExpirationPolicy.class);
-        kryo.register(RememberMeDelegatingExpirationPolicy.class);
-        kryo.register(TimeoutExpirationPolicy.class);
-        kryo.register(HardTimeoutExpirationPolicy.class);
-        kryo.register(AlwaysExpiresExpirationPolicy.class);
-        kryo.register(ThrottledUseAndTimeoutExpirationPolicy.class);
-        kryo.register(TicketGrantingTicketExpirationPolicy.class);
-        kryo.register(BaseDelegatingExpirationPolicy.class);
-    }
-
-    private static void registerMessageDescriptorsWithKryo(final CloseableKryo kryo) {
-        kryo.register(DefaultMessageDescriptor.class);
-        kryo.register(PasswordExpiringWarningMessageDescriptor.class);
-    }
-
-    private static void registerCasServicesAuthenticationPolicy(final CloseableKryo kryo) {
-        kryo.register(AnyAuthenticationHandlerRegisteredServiceAuthenticationPolicyCriteria.class);
-        kryo.register(AllAuthenticationHandlersRegisteredServiceAuthenticationPolicyCriteria.class);
-        kryo.register(AllowedAuthenticationHandlersRegisteredServiceAuthenticationPolicyCriteria.class);
-        kryo.register(GroovyRegisteredServiceAuthenticationPolicyCriteria.class);
-        kryo.register(NotPreventedRegisteredServiceAuthenticationPolicyCriteria.class);
-        kryo.register(RestfulRegisteredServiceAuthenticationPolicyCriteria.class);
-    }
 }
diff --git a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/RegisteredServiceSerializer.java b/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/RegisteredServiceSerializer.java
deleted file mode 100644
index 45a3b4a4398..00000000000
--- a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/RegisteredServiceSerializer.java
+++ /dev/null
@@ -1,147 +0,0 @@
-package org.apereo.cas.memcached.kryo.serial;
-
-import org.apereo.cas.services.DefaultRegisteredServiceAccessStrategy;
-import org.apereo.cas.services.DefaultRegisteredServiceAuthenticationPolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceMultifactorPolicy;
-import org.apereo.cas.services.DefaultRegisteredServiceUsernameProvider;
-import org.apereo.cas.services.RefuseRegisteredServiceProxyPolicy;
-import org.apereo.cas.services.RegexRegisteredService;
-import org.apereo.cas.services.RegisteredService;
-import org.apereo.cas.services.RegisteredServiceAccessStrategy;
-import org.apereo.cas.services.RegisteredServiceAttributeReleasePolicy;
-import org.apereo.cas.services.RegisteredServiceAuthenticationPolicy;
-import org.apereo.cas.services.RegisteredServiceLogoutType;
-import org.apereo.cas.services.RegisteredServiceMultifactorPolicy;
-import org.apereo.cas.services.RegisteredServiceProxyPolicy;
-import org.apereo.cas.services.RegisteredServicePublicKey;
-import org.apereo.cas.services.RegisteredServicePublicKeyImpl;
-import org.apereo.cas.services.RegisteredServiceUsernameAttributeProvider;
-import org.apereo.cas.services.ReturnAllowedAttributeReleasePolicy;
-
-import com.esotericsoftware.kryo.Kryo;
-import com.esotericsoftware.kryo.Serializer;
-import com.esotericsoftware.kryo.io.Input;
-import com.esotericsoftware.kryo.io.Output;
-import lombok.SneakyThrows;
-import lombok.val;
-import org.apache.commons.lang3.ObjectUtils;
-import org.apache.commons.lang3.StringUtils;
-
-import java.net.URL;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-
-/**
- * Serializer for {@link RegisteredService} instances.
- *
- * @author Misagh Moayyed
- * @since 4.1.0
- */
-public class RegisteredServiceSerializer extends Serializer<RegisteredService> {
-
-    /**
-     * In case the url object is null in the service,
-     * we need to be able to return a default/mock url.
-     *
-     * @return mock url
-     */
-    @SneakyThrows
-    private static URL getEmptyUrl() {
-        return new URL("https://");
-    }
-
-    /**
-     * Write object by reflection.
-     *
-     * @param kryo   the kryo
-     * @param output the output
-     * @param obj    the obj
-     */
-    private static void writeObjectByReflection(final Kryo kryo, final Output output, final Object obj) {
-        val className = obj.getClass().getCanonicalName();
-        kryo.writeObject(output, className);
-        kryo.writeObject(output, obj);
-    }
-
-    /**
-     * Read object by reflection.
-     *
-     * @param <T>   the type parameter
-     * @param kryo  the kryo
-     * @param input the input
-     * @param clazz the clazz
-     * @return the t
-     */
-    @SneakyThrows
-    private static <T> T readObjectByReflection(final Kryo kryo, final Input input, final Class<T> clazz) {
-        val className = kryo.readObject(input, String.class);
-        val foundClass = (Class<T>) Class.forName(className);
-        val result = kryo.readObject(input, foundClass);
-
-        if (!clazz.isAssignableFrom(result.getClass())) {
-            throw new ClassCastException("Result [" + result
-                + " is of type " + result.getClass()
-                + " when we were expecting " + clazz);
-        }
-        return result;
-    }
-
-    @Override
-    public void write(final Kryo kryo, final Output output, final RegisteredService service) {
-        kryo.writeObject(output, service.getServiceId());
-        kryo.writeObject(output, StringUtils.defaultIfEmpty(service.getName(), StringUtils.EMPTY));
-        kryo.writeObject(output, StringUtils.defaultIfEmpty(service.getDescription(), StringUtils.EMPTY));
-        kryo.writeObject(output, service.getId());
-        kryo.writeObject(output, service.getEvaluationOrder());
-        kryo.writeObject(output, ObjectUtils.defaultIfNull(service.getLogo(), getEmptyUrl()));
-        kryo.writeObject(output, service.getLogoutType());
-        kryo.writeObject(output, ObjectUtils.defaultIfNull(service.getLogoutUrl(), StringUtils.EMPTY));
-        
-        kryo.writeObject(output, StringUtils.defaultIfEmpty(service.getTheme(), StringUtils.EMPTY));
-        kryo.writeObject(output, StringUtils.defaultIfEmpty(service.getResponseType(), StringUtils.EMPTY));
-
-        writeObjectByReflection(kryo, output, ObjectUtils.defaultIfNull(service.getPublicKey(), new RegisteredServicePublicKeyImpl()));
-        writeObjectByReflection(kryo, output, ObjectUtils.defaultIfNull(service.getProxyPolicy(), new RefuseRegisteredServiceProxyPolicy()));
-        writeObjectByReflection(kryo, output, ObjectUtils.defaultIfNull(service.getAttributeReleasePolicy(), new ReturnAllowedAttributeReleasePolicy()));
-        writeObjectByReflection(kryo, output, ObjectUtils.defaultIfNull(service.getUsernameAttributeProvider(), new DefaultRegisteredServiceUsernameProvider()));
-        writeObjectByReflection(kryo, output, ObjectUtils.defaultIfNull(service.getAccessStrategy(), new DefaultRegisteredServiceAccessStrategy()));
-        writeObjectByReflection(kryo, output, ObjectUtils.defaultIfNull(service.getMultifactorPolicy(), new DefaultRegisteredServiceMultifactorPolicy()));
-        writeObjectByReflection(kryo, output, ObjectUtils.defaultIfNull(service.getAuthenticationPolicy(), new DefaultRegisteredServiceAuthenticationPolicy()));
-        writeObjectByReflection(kryo, output, ObjectUtils.defaultIfNull(service.getContacts(), new ArrayList<>(0)));
-
-        kryo.writeObject(output, StringUtils.defaultIfEmpty(service.getInformationUrl(), StringUtils.EMPTY));
-        kryo.writeObject(output, StringUtils.defaultIfEmpty(service.getPrivacyUrl(), StringUtils.EMPTY));
-        kryo.writeObject(output, new HashMap<>(service.getProperties()));
-    }
-
-    @Override
-    public RegisteredService read(final Kryo kryo, final Input input, final Class<RegisteredService> type) {
-        val svc = new RegexRegisteredService();
-        svc.setServiceId(kryo.readObject(input, String.class));
-        svc.setName(kryo.readObject(input, String.class));
-        svc.setDescription(kryo.readObject(input, String.class));
-        svc.setId(kryo.readObject(input, Long.class));
-        svc.setEvaluationOrder(kryo.readObject(input, Integer.class));
-        svc.setLogo(kryo.readObject(input, String.class));
-        svc.setLogoutType(kryo.readObject(input, RegisteredServiceLogoutType.class));
-        svc.setLogoutUrl(kryo.readObject(input, String.class));
-       
-        svc.setTheme(kryo.readObject(input, String.class));
-        svc.setResponseType(StringUtils.defaultIfBlank(kryo.readObject(input, String.class), null));
-
-        svc.setPublicKey(readObjectByReflection(kryo, input, RegisteredServicePublicKey.class));
-        svc.setProxyPolicy(readObjectByReflection(kryo, input, RegisteredServiceProxyPolicy.class));
-        svc.setAttributeReleasePolicy(readObjectByReflection(kryo, input, RegisteredServiceAttributeReleasePolicy.class));
-        svc.setUsernameAttributeProvider(readObjectByReflection(kryo, input, RegisteredServiceUsernameAttributeProvider.class));
-        svc.setAccessStrategy(readObjectByReflection(kryo, input, RegisteredServiceAccessStrategy.class));
-        svc.setMultifactorPolicy(readObjectByReflection(kryo, input, RegisteredServiceMultifactorPolicy.class));
-        svc.setAuthenticationPolicy(readObjectByReflection(kryo, input, RegisteredServiceAuthenticationPolicy.class));
-        svc.setContacts(readObjectByReflection(kryo, input, List.class));
-
-        svc.setInformationUrl(StringUtils.defaultIfBlank(kryo.readObject(input, String.class), null));
-        svc.setPrivacyUrl(StringUtils.defaultIfBlank(kryo.readObject(input, String.class), null));
-        svc.setProperties(kryo.readObject(input, HashMap.class));
-        return svc;
-    }
-}
diff --git a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/SimpleWebApplicationServiceSerializer.java b/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/SimpleWebApplicationServiceSerializer.java
deleted file mode 100644
index 2c5e81f1f74..00000000000
--- a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/SimpleWebApplicationServiceSerializer.java
+++ /dev/null
@@ -1,30 +0,0 @@
-package org.apereo.cas.memcached.kryo.serial;
-
-import org.apereo.cas.authentication.principal.SimpleWebApplicationServiceImpl;
-import org.apereo.cas.authentication.principal.WebApplicationServiceFactory;
-
-import com.esotericsoftware.kryo.Kryo;
-import com.esotericsoftware.kryo.Serializer;
-import com.esotericsoftware.kryo.io.Input;
-import com.esotericsoftware.kryo.io.Output;
-import lombok.val;
-
-/**
- * Serializer for {@link SimpleWebApplicationServiceImpl} class.
- *
- * @author Marvin S. Addison
- * @since 3.0.0
- */
-public class SimpleWebApplicationServiceSerializer extends Serializer<SimpleWebApplicationServiceImpl> {
-
-    @Override
-    public void write(final Kryo kryo, final Output output, final SimpleWebApplicationServiceImpl service) {
-        kryo.writeObject(output, service.getId());
-    }
-
-    @Override
-    public SimpleWebApplicationServiceImpl read(final Kryo kryo, final Input input, final Class<SimpleWebApplicationServiceImpl> type) {
-        val id = kryo.readObject(input, String.class);
-        return new WebApplicationServiceFactory().createService(id, SimpleWebApplicationServiceImpl.class);
-    }
-}
diff --git a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/ThrowableSerializer.java b/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/ThrowableSerializer.java
index 728f513d188..8349d792c6a 100644
--- a/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/ThrowableSerializer.java
+++ b/support/cas-server-support-memcached-core/src/main/java/org/apereo/cas/memcached/kryo/serial/ThrowableSerializer.java
@@ -1,5 +1,7 @@
 package org.apereo.cas.memcached.kryo.serial;
 
+import org.apereo.cas.util.LoggingUtils;
+
 import com.esotericsoftware.kryo.Kryo;
 import com.esotericsoftware.kryo.Serializer;
 import com.esotericsoftware.kryo.io.Input;
@@ -31,7 +33,7 @@ public class ThrowableSerializer extends Serializer<Throwable> {
             val msg = kryo.readObject(input, String.class);
             return (Throwable) clazz.getDeclaredConstructor(String.class).newInstance(msg);
         } catch (final Exception e) {
-            LOGGER.error(e.getMessage(), e);
+            LoggingUtils.error(LOGGER, e);
         }
         return new Throwable();
     }
jbond added a subscriber: elukey.Feb 4 2021, 5:10 PM
In T273867#6804120, @jbond wrote:

Testing different transcoders shows this is only an issue with KYRO:

cas eversiontranscoderstatus
6.3.1KRYOfail
6.4.0-RC1KRYOfail
6.4.0-RC1SERIALworks
6.3.1SERIALworks
6.4.0-RC1WHALINworks
6.3.1WHALINworks
6.4.0-RC1WHALINV1works
6.3.1WHALINV1works

@elukey this is probably CAS or java specific but just in case i wonder if you know what the difference between theses transcoders are?

elukey added a subscriber: Gehel.Feb 4 2021, 5:35 PM

Probably best to ask to @Gehel :)

Marostegui added a comment.Feb 5 2021, 11:08 AM

Thanks @jbond. However, I still get logged out from places like tendril, several times per day - today I think I got logged out twice already :-(
Icinga though seems to be working better today

jbond added a comment.EditedFeb 5 2021, 11:40 AM
In T273867#6806145, @Marostegui wrote:

Thanks @jbond. However, I still get logged out from places like tendril, several times per day - today I think I got logged out twice already :-(
Icinga though seems to be working better today

from the logs it looks like you are not ticking the rember me box. By design the session will only be valid for one hour or with the session which ever comes up first unless one ticks remember me

sudo grep -A5 'WHO: Marostegui' /var/log/cas/cas.log
WHO: Marostegui
WHAT: Supplied credentials: [RememberMeUsernamePasswordCredential(super=UsernamePasswordCredential(username=Marostegui, source=null, customFields={}), rememberMe=false)]
ACTION: AUTHENTICATION_SUCCESS
APPLICATION: CAS
WHEN: Fri Feb 05 06:26:13 UTC 2021

We see this about every hour give our take

WHO: Marostegui
WHAT: Supplied credentials: [RememberMeUsernamePasswordCredential(super=UsernamePasswordCredential(username=Marostegui, source=null, customFields={}), rememberMe=false)]
ACTION: AUTHENTICATION_SUCCESS
APPLICATION: CAS
WHEN: Fri Feb 05 10:28:28 UTC 2021
Marostegui added a comment.Feb 5 2021, 11:42 AM

mmm, maybe it is a different browser. I will check!
Thanks

Gehel added a project: Discovery-Search.Feb 7 2021, 8:52 AM
MPhamWMF moved this task from needs triage to Current work on the Discovery-Search board.Feb 8 2021, 4:20 PM
MPhamWMF edited projects, added Discovery-Search (Current work); removed Discovery-Search.
MPhamWMF set the point value for this task to 3.Feb 8 2021, 4:47 PM
MPhamWMF moved this task from Incoming to Ready for Development on the Discovery-Search (Current work) board.
Zbyszko moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board.Feb 9 2021, 8:08 AM
Zbyszko added a subscriber: Zbyszko.Feb 9 2021, 8:22 AM

@jbond About Kryo - from what I see in Kryo src:

 	/** Registers the class using the lowest, next available integer ID and the {@link Kryo#getDefaultSerializer(Class) default
	 * serializer}. If the class is already registered, no change will be made and the existing registration will be returned.
	 * Registering a primitive also affects the corresponding primitive wrapper.
	 * <p>
	 * Because the ID assigned is affected by the IDs registered before it, the order classes are registered is important when
	 * using this method. The order must be the same at deserialization as it was for serialization. */

public Registration register (Class type) {

order of registering classes is important to Kryo and the change in registered classes affected that. If you want to use the same state, new registered classes have to be added at the end (or with a specific id)
In general, Kryo isn't really good for a model evolution - we have similiar issues with it in WDQS streaming updater. I can't unfortunately speak about other serializers, since I'm not sure what those names represent.

jbond added a comment.Feb 9 2021, 1:38 PM

@Zbyszko Thanks for looking into this its really appreciated

we have similiar issues with it in WDQS streaming updater

is it refering to Kyro here? i had thought that was a apereo cas specific thing . if its more general that's useful to know.

can't unfortunately speak about other serializers, since I'm not sure what those names represent.

I think theses are more generic memcached serialises. i can run some tests to see what difference this makes on the value blob

Other then switching serialises where you able to spot any obvious places in the code where a fix may be applied. Failing this do you have an idea of how much effort it would take to create a unit test to reproduce this error. This would allow use to send a more substantial report upstream.

Zbyszko added a comment.Feb 9 2021, 2:07 PM
In T273867#6814932, @jbond wrote:

@Zbyszko Thanks for looking into this its really appreciated

we have similiar issues with it in WDQS streaming updater

is it refering to Kyro here? i had thought that was a apereo cas specific thing . if its more general that's useful to know.

Yeah, it's about Kryo - although a bit different issue than here from what I see.

can't unfortunately speak about other serializers, since I'm not sure what those names represent.

I think theses are more generic memcached serialises. i can run some tests to see what difference this makes on the value blob

Other then switching serialises where you able to spot any obvious places in the code where a fix may be applied. Failing this do you have an idea of how much effort it would take to create a unit test to reproduce this error. This would allow use to send a more substantial report upstream.

I'd try to reorganize register() methods on kryo - so that new ones are after the old ones. I'm not deep enough to understand the full issue here - but from looking a bit further, exception correlates with the javadoc I posted from the source code.

jbond added a comment.Feb 9 2021, 2:09 PM
In T273867#6815032, @Zbyszko wrote:

I'd try to reorganize register() methods on kryo - so that new ones are after the old ones. I'm not deep enough to understand the full issue here - but from looking a bit further, exception correlates with the javadoc I posted from the source code.

@Zbyszko Thanks for the effort ill send this upstream <3

Zbyszko added a comment.Feb 9 2021, 2:21 PM
In T273867#6815035, @jbond wrote:
In T273867#6815032, @Zbyszko wrote:

I'd try to reorganize register() methods on kryo - so that new ones are after the old ones. I'm not deep enough to understand the full issue here - but from looking a bit further, exception correlates with the javadoc I posted from the source code.

@Zbyszko Thanks for the effort ill send this upstream <3

You're welcome! Let me know if there's something here, especially Java related, I can help with.

jbond added a comment.Feb 9 2021, 3:05 PM

Thanks i sent you response upstream will see what they come back with
https://groups.google.com/u/1/a/apereo.org/g/cas-user/c/MkpgAZZn-Mw

• crusnov added a subscriber: • crusnov.Feb 10 2021, 5:31 PM
Dzahn added a subscriber: Dzahn.Feb 11 2021, 7:49 PM
Zbyszko moved this task from In Progress to Waiting on the Discovery-Search (Current work) board.Feb 15 2021, 3:05 PM
Gehel removed a project: Discovery-Search (Current work).Feb 15 2021, 4:44 PM

Removing discovery-search, if you need our help again, please ping us!

jbond added a comment.Mar 10 2021, 3:32 PM

I have started to look at this again and wanted to look at the difference between the KRYO vs other encodings and noticed that when i try to use the following python script

#!/usr/bin/python3

import sys
import memcache
import os
import pickle
import subprocess
import argparse


def main():
    server = "127.0.0.1:11000"
    connection = memcache.Client([server])
    try:
        memcdump = subprocess.run(["/usr/bin/memcdump", "--servers=" + server], capture_output=True)
    except subprocess.CalledProcessError as e:
        print("memcache dump failed: {}".format(e))
        return 1

    for memkey in memcdump.stdout.decode("utf-8").split('\n'):
        if memkey:
            memvalue = connection.get(memkey)
            print(f'{memkey}: {memvalue}')


if __name__ == "__main__":

I receive the following error

Traceback (most recent call last):
  File "./memcached.py", line 27, in <module>
    raise SystemExit(main())
  File "./memcached.py", line 22, in main
    memvalue = connection.get(memkey)
  File "/usr/lib/python3/dist-packages/memcache.py", line 1121, in get
    return self._get('get', key)
  File "/usr/lib/python3/dist-packages/memcache.py", line 1105, in _get
    return _unsafe_get()
  File "/usr/lib/python3/dist-packages/memcache.py", line 1093, in _unsafe_get
    value = self._recv_value(server, flags, rlen)
  File "/usr/lib/python3/dist-packages/memcache.py", line 1257, in _recv_value
    buf = self.decompressor(buf)
zlib.error: Error -3 while decompressing data: incorrect header check

If i dump the raw value earlier i notice it has the following start sequence: '\xac\xed\x00\x05sr\x00'. however in production we see '#\x10-\x06\x03'. further idp-test also seems to have a lot of additional null data

lMetaDatat\x002Lorg/apereo/cas/authentication/CredentialMetaData;L\x00\x0bhandlerNameq\x00~\x00\nL\x00\tprincipalq\x00~\x00)L\x00\x08warningsq\x00~\x00(xpsq\x00~\x00Kq\x00~\x00Qq\x00~\x00Rq\x00~\x00Bsq\x00~\x00Ysq\x00~\x00\x1a?@\x00\x00\x00\x00\x00\x0cw\x08\x00\x00\x00\x10\x00\x00\x00\x05q\x00~\x00\\sq\x00~\x00-\x00\x00\x00\x01w\x04\x00\x00\x00\x01q\x00~\x00^xq\x00~\x00vsq\x00~\x00-\x00\x00\x00\x01w\x04\x00\x00\x00\x01q\x00~\x00xxq\x00~\x00_sq\x00~\x00-\x00\x00\x00\x01w\x04\x00\x00\x00\x01q\x00~\x00axq\x00~\x00bsq\x00~\x00-\x00\x00\x00\x12w\x04\x00\x00\x00\x12q\x00~\x00dq\x00~\x00eq\x00~\x00fq\x00~\x00gq\x00~\x00hq\x00~\x00iq\x00~\x00jq\x00~\x00kq\x00~\x00lq\x00~\x00mq\x00~\x00nq\x00~\x00oq\x00~\x00pq\x00~\x00qq\x00~\x00rq\x00~\x00sq\x00~\x00tq\x00~\x00uxq\x00~\x00ysq\x00~\x00-\x00\x00\x00\x01w\x04\x00\x00\x00\x01q\x00~\x00{xxq\x00~\x00Rsq\x00~\x00-\x00\x00\x00\x00w\x04\x00\x00\x00\x00xq\x00~\x00Csq\x00~\x00}sq\x00~\x00Kq\x00~\x00Vq\x00~\x00Wq\x00~\x00Csq\x00~\x00Ysq\x00~\x00\x1a?@\x00\x00\x00\x00\x00\x01w\x08\x00\x00\x00\x01\x00\x00\x00\x00xq\x00~\x00Rsq\x00~\x00-\x00\x00\x00\x00w\x04\x00\x00\x00\x00xx\x00sq\x00~\x00-\x00\x00\x00\x00w\x04\x00\x00\x00\x00xsr\x00\x11java.util.HashSet\xbaD\x85\x95\x96\xb8\xb74\x03\x00\x00xpw\x0c\x00\x00\x00\x01?@\x00\x00\x00\x00\x00\x00xpsq\x00~\x00\x1a?@\x00\x00\x00\x00\x00\x01w\x08\x00\x00\x00\x01\x00\x00\x00\x00xsq\x00~\x00\x1a?@\x00\x00\x00\x00\x00\x01w\x08\x00\x00\x00\x01\x00\x00\x00\x00xp\r\n'

One thing to not here is that it seems the standard python memcached library cant read entries that have been written to using cas 6.3.1 and the KYRO encoder

jbond added a comment.Mar 10 2021, 4:04 PM

I have just built 6.3.2 and python can at least read the value without the above stack trace however cas still has trouble retrieving the value from memcache although now with a slightly different error :

java.util.concurrent.ExecutionException: com.esotericsoftware.kryo.KryoException: com.esotericsoftware.kryo.KryoException: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
 ERROR [org.apereo.cas.ticket.registry.MemcachedTicketRegistry] - <Exception waiting for value>
java.lang.RuntimeException: Exception waiting for value
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1236) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1256) ~[spymemcached-2.12.3.jar:2.12.3]
        at org.apereo.cas.ticket.registry.MemcachedTicketRegistry.getTicket(MemcachedTicketRegistry.java:96) ~[cas-server-support-memcached-ticket-registry-6.3.2.jar:6.3.2]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:51) ~[cas-server-core-tickets-api-6.3.2.jar:6.3.2]
        at org.apereo.cas.ticket.registry.AbstractTicketRegistry.getTicket(AbstractTicketRegistry.java:63) ~[cas-server-core-tickets-api-6.3.2.jar:6.3.2]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy135.getTicket(Unknown Source) ~[?:?]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getTicketGrantingTicket(DefaultTicketRegistrySupport.java:45) ~[cas-server-core-tickets-api-6.3.2.jar:6.3.2]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getAuthenticationFrom(DefaultTicketRegistrySupport.java:54) ~[cas-server-core-tickets-api-6.3.2.jar:6.3.2]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport.getAuthenticatedPrincipalFrom(DefaultTicketRegistrySupport.java:60) ~[cas-server-core-tickets-api-6.3.2.jar:6.3.2]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$FastClassBySpringCGLIB$$885171cb.invoke(<generated>) ~[cas-server-core-tickets-api-6.3.2.jar:6.3.2]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367) ~[spring-tx-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118) ~[spring-tx-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apereo.cas.ticket.registry.DefaultTicketRegistrySupport$$EnhancerBySpringCGLIB$$85e9e23f.getAuthenticatedPrincipalFrom(<generated>) ~[cas-server-core-tickets-api-6.3.2.jar:6.3.2]
        at org.apereo.cas.logging.web.ThreadContextMDCServletFilter.doFilter(ThreadContextMDCServletFilter.java:94) ~[cas-server-core-logging-6.3.2.jar:6.3.2]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apereo.inspektr.common.web.ClientInfoThreadLocalFilter.doFilter(ClientInfoThreadLocalFilter.java:66) ~[inspektr-common-1.8.10.GA.jar:1.8.10.GA]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) ~[spring-boot-2.3.7.RELEASE.jar:2.3.7.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71) ~[log4j-web-2.14.0.jar:2.14.0]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:688) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) ~[tomcat9-catalina-9.0.31.jar:9.0.31]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat9-coyote-9.0.31.jar:9.0.31]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat9-util-9.0.31.jar:9.0.31]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.util.concurrent.ExecutionException: com.esotericsoftware.kryo.KryoException: com.esotericsoftware.kryo.KryoException: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
Serialization trace:
args (org.apereo.cas.authentication.exceptions.MixedPrincipalException)
filters (org.apereo.cas.services.support.RegisteredServiceChainingAttributeFilter)
        at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
        at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:97) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3]
        ... 65 more
Caused by: com.esotericsoftware.kryo.KryoException: com.esotericsoftware.kryo.KryoException: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
Serialization trace:
args (org.apereo.cas.authentication.exceptions.MixedPrincipalException)
filters (org.apereo.cas.services.support.RegisteredServiceChainingAttributeFilter)
        at org.apereo.cas.memcached.kryo.CasKryoTranscoder.decode(CasKryoTranscoder.java:74) ~[cas-server-support-memcached-core-6.3.2.jar:6.3.2]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy254.decode(Unknown Source) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$1.call(TranscodeService.java:63) ~[spymemcached-2.12.3.jar:2.12.3]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.run(TranscodeService.java:110) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:96) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3]
        ... 65 more
Caused by: com.esotericsoftware.kryo.KryoException: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
Serialization trace:
args (org.apereo.cas.authentication.exceptions.MixedPrincipalException)
filters (org.apereo.cas.services.support.RegisteredServiceChainingAttributeFilter)
        at com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:147) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:124) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:732) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:125) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:124) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813) ~[kryo-5.0.2.jar:?]
        at org.apereo.cas.memcached.kryo.CasKryoTranscoder.decode(CasKryoTranscoder.java:72) ~[cas-server-support-memcached-core-6.3.2.jar:6.3.2]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy254.decode(Unknown Source) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$1.call(TranscodeService.java:63) ~[spymemcached-2.12.3.jar:2.12.3]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.run(TranscodeService.java:110) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:96) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3]
        ... 65 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
        at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:211) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:42) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:732) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:125) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:124) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:732) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.ReflectField.read(ReflectField.java:125) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:124) ~[kryo-5.0.2.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813) ~[kryo-5.0.2.jar:?]
        at org.apereo.cas.memcached.kryo.CasKryoTranscoder.decode(CasKryoTranscoder.java:72) ~[cas-server-support-memcached-core-6.3.2.jar:6.3.2]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.cloud.context.scope.GenericScope$LockedScopedProxyFactoryBean.invoke(GenericScope.java:499) ~[spring-cloud-context-2.2.6.RELEASE.jar:2.2.6.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.12.RELEASE.jar:5.2.12.RELEASE]
        at com.sun.proxy.$Proxy254.decode(Unknown Source) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$1.call(TranscodeService.java:63) ~[spymemcached-2.12.3.jar:2.12.3]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at net.spy.memcached.transcoders.TranscodeService$Task.run(TranscodeService.java:110) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.transcoders.TranscodeService$Task.get(TranscodeService.java:96) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.internal.GetFuture.get(GetFuture.java:70) ~[spymemcached-2.12.3.jar:2.12.3]
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1229) ~[spymemcached-2.12.3.jar:2.12.3
jbond added a comment.Mar 15 2021, 10:51 AM

See below for a table showing the various value size for each encoding tested on 6.2.7, theses are vary rough measures simply len(value) on the raw buffer received from memcache however i think its enough to demonstrate that KYRO is superior and there dosn;t seem to be much in it for the others. however even with the worst estimates 1GB of ram still allows us to support ~250k sessions

TranscoderSize
KYRO1472
SERIAL4361
WHALIN4283
WHALINV14361
jbond added a comment.Mar 15 2021, 10:55 AM

I just did a quick test and the value size on idp-test1001 with SERIAL is 6208 and the size on idp1001 using KYRO is 1919

gerritbot added a comment.Mar 16 2021, 10:07 AM

Change 672679 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Switch the IDPs to the serial Memcached transcoder

https://gerrit.wikimedia.org/r/672679

gerritbot added a comment.Mar 17 2021, 11:06 AM

Change 672679 merged by Muehlenhoff:
[operations/puppet@production] Switch the IDPs to the serial Memcached transcoder

https://gerrit.wikimedia.org/r/672679

MoritzMuehlenhoff renamed this task from Investigate CAS Session logout to Kryo memcached transcoder broken in CAS 6.3/6.4.May 20 2021, 10:28 AM
MoritzMuehlenhoff lowered the priority of this task from Medium to Low.
MoritzMuehlenhoff updated the task description. (Show Details)
MoritzMuehlenhoff removed the point value for this task.
gerritbot added a comment.May 27 2021, 10:16 AM

Change 661713 merged by Jbond:

[operations/puppet@production] apereo_cas: rename config properties

https://gerrit.wikimedia.org/r/661713

Aklapper added a project: Infrastructure-Foundations.Jun 21 2021, 9:00 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL