Temporary accounts become permanently unused after 90 days or logout, leaving behind occasional user and talk pages. A user who comes across these pages may not understand that the temporary account has expired. To mitigate this, we should indicate the expiry of the temp user on these pages.
TBD
The other aspect that is implied in the title but isn't explicit is when the TA is deliberately logged out of.
I think we should probably go further and simply disable editing talk pages in the same way as MediaWiki namespace pages (which can be edited by admins and interface admins). You're never going to hear from that temporary account again. There are some edge cases I guess, but it's not a win for anyone to spend time on them I think (one I can think of is a user has a TA for a long period and then moves to a new TA at 90 days and in general has a stable TA, they may want to be able to point from the old page to the new). And in those edge cases, I would say those TAs are sufficiently intelligent to know how to request help (from an admin).
I am not personally certain a protection should be issued for the main user page, but I think a notification would be both fair and desirable.
simply disable editing talk pages in the same way as MediaWiki namespace pages
If a temporary account added spam to their own user/talk page, non-admins should still be able to remove them even if the temporary account is expired.