If Pyramid sessions are one-way hashed and not stored server-side, where is the data coming from?
The cookie contains all of the data. The session content itself is stored in the cookie, alongside an hmac signature of that content. This means that a client may view the content if they try hard enough, but they cannot change it as they cannot create trusted signatures without the server-side secret.

How is it possible to remain secure with browser-scoped sessions unless every session cookie the user has recently been issued is somehow remembered and blacklisted/invalidated?

It depends what you're using the session for - you need to take these issues into consideration before putting data into a session object.

Is it possible then to invalidate sessions server-side, a little more robust than just the headers=forget(request) pattern?

Not unless you store some sort of id in the session and then a server-side table of blacklisted ids. If you did this then you could write your own wrapper session-factory that opened the session, checked the id, and returned an empty one if it was blacklisted. Of course then you might just want to use a server-side session library like pyramid_redis_sessions or pyramid_beaker with one of its server-side-storage backends.
When using the SignedCookieSessionFactory, the documentation states that the sha512 HMAC digest algorithm is used. As a result of this, once session data is serialised, it is signed and sent to the user's client under the session cookie.
There is no mention in Pyramid's documentation that sessions are also cached server-side (under this SessionFactory).
This presents a contradiction and has also led to authentication confusion when paired with SessionAuthenticationPolicy. If session data cannot be retrieved from the client's session cookie (as it is one-way hashed), how is it that the following is possible?

Authenticate with the application such that reading Request.authenticated_userid does not return None.
Copy the session cookie to the clipboard.
Logout by accessing a view whereby the headers from forget(request) are returned in the Response.
Replace the session cookie with the copied value.
The user is now logged back in.

I understand that simply deleting the cookie client-side is insufficient to completely invalidate the session (without blacklisting), but this then poses the following questions:

How is it possible to remain secure with browser-scoped sessions unless every session cookie the user has recently been issued is somehow remembered and blacklisted/invalidated?
Is the session cookie actually a key/session ID that's being used to lookup a session in memory? As it's one-way signed, surely this is the only explanation?
Is it possible then to invalidate sessions server-side, a little more robust than just the headers=forget(request) pattern?

Ultimately, I imagine the answer is something along the lines of:
"Maintain server-side sessions using an include such as pyramid_redis_sessions"
Any explanation would be appreciated.