Back to the Salt Mine(s)

What does the idiom “Back to the Salt Mine(s)” mean?

Definition:

We have to go back to work.

For example:

“Thanks for having lunch with me. We should do it again. Now, back to the salt mine.”

Notes:

This is often used by itself, as in the example.