TimeSync: Whonix Time Synchronization Mechanism

From Whonix
< Dev
Jump to navigation Jump to search

Design Documentation of Whonix Time Synchronization Mechanism

Basic Knowledge[edit]

Introduction[edit]

The Post Install Advice is part of this design, therefore read Network Time Syncing first. Also read Network Time Synchronization as well.

Local Clock Leaks[edit]

  • TCP Sequence Numbers → tirdadarchive.org [1]
  • ICMP timestamps → blocked by Whonix firewall for Whonix VMs. Host: Recommendation to disable on the host.
  • TCP timestamps → blocked in Whonix VMs by security-miscarchive.org.; Recommendation to disable on the host.
  • NTP [2] [3] → Not installed by default in Whonix. Host: TODO
  • Automatic updaters and other network using cron jobs that run at non-random, canned [4] times.
  • Javascript. [5] [6]
  • E-mail clients such as Mozilla Thunderbird and others. [7] [8] [9] [10]
  • https / ssl, older [11] versions of openssl, TLS HELLO gmt_unix_time [12] What about other implementations of SSL / TLS, other than openssl?
  • older [11] versions of Tor (NETINFO) [13]
  • web servers (time stamps from http headers (RFC 2616)) [14]

Virtualizers Time Synchronization Features[edit]

VirtualBox Time Synchronization Features[edit]

For understanding Whonix Time Synchronization Mechanism, it is required to know VirtualBox's time synchronization features:

  • VirtualBox uses the host's time if it needs to correct the time for guests.
  • (Some of VirtualBox's time synchronization features depend on guest additions.)
  • By default VirtualBox corrects Virtual Machine guests' virtual hardware system clock,
    • when they get powered on,
    • when they resume from suspension and
    • when their clock is more than X minutes off.

KVM's Time Synchronization Features[edit]

TODO: document

Qubes Time Synchronization Features[edit]

TODO: document

Xen DomU’s get their initial time from Xen (Qubes dom0) at VM start.

  • qvm-sync-clock.service
  • qvm-sync-clock.timer

qvm-sync-clock gets time from Qubes ClockVM.

It is used because otherwise Xen DomU’s would start to clock drift more and more. This this answer by marmarekarchive.org.

qvm-sync-clock is unwanted in sys-whonix and anon-whonix because sdwdate runs there.

qvm-sync-clock is disabled in Qubes-Whonix™ Templates until version Qubes-Whonix™ 16.0.3.7. To be re-considered for later versions. Qubes-Whonix™ get their time from dom0 at VM startup, which is then randomized using Boot Clock Randomization.

Future: qvm-sync-clock should be equally safe to run inside Qubes-Whonix Templates, if passed though clock-random-manual-cli.

When is /etc/qubes-rpc/qubes.SetDateTime run?

sdwdate Time Sources Criteria[edit]

See sdwdate Time Sources Criteria.

Facts[edit]

People have been using:

  • Tor Browser and Mozilla Firefox in parallel
  • Tails inside VMs and Firefox on the host
  • Tor Browser inside Whonix-Workstation™ and Mozilla Firefox on the host

Other Facts[edit]

  • Almost no one and no operating system is using Secure Network Time Synchronization or External Clocks by default. Most systems synchronize the system clock using unauthenticated NTP. An adversary tampering with NTP replies or malicious NTP server makes things even worse. Even if there was authenticated NTP, there is a requirement for a distributed trust model.
  • A system not using NTP or using authenticated NTP stands out from most other users.
  • clock jumps are bad: wired: The Inside Story of the Extra Second That Crashed the Webarchive.org

Definitions[edit]

  • Time Sanity Check in context of Whonix: Is an init.d script, which checks if the system clock is between build timestamp and expiration date (17 MAY 2033 10:00:00).

Attacks[edit]

A correct system clock is crucial for many reasons (see footnotes for more):

  • Attack 1: Replay attacks [15]
  • Attack 2: Feeding old/outdated/known vulnerable updates and (https) certificates [16]
  • Attack 3: Deanonymizing [17]
  • Attack 4: Linking all sessions to the same pseudonym [18]
  • Attack 5: Locating onion services. [19]

Goals[edit]

Completely Isolated Virtual Time[edit]

Quote https://www.kernel.org/doc/Documentation/virtual/kvm/timekeeping.txtarchive.org

4.8) Covert channels and leaks

In addition to the above problems, time information will inevitably leak to the guest about the host in anything but a perfect implementation of virtualized time. This may allow the guest to infer the presence of a hypervisor (as in a red-pill type detection), and it may allow information to leak between guest by using CPU utilization itself as a signalling channel. Preventing such problems would require completely isolated virtual time which may not track real time any longer. This may be useful in certain security or QA contexts, but in general is not recommended for real-world deployment scenarios.

Therefore we want a completely isolated virtual time.

Whonix Time Synchronization Mechanism[edit]

Overview[edit]

  • Whonix leaves the host's system clock or time synchronization mechanism untouched.
  • Whonix-Gateway™ and Whonix-Workstation are commonly used inside virtualizers and therefore have their own virtual hardware system clocks.
  • Virtualizers time synchronization get disabled by Whonix.
  • VM Settings, Hardware Clock: set to UTC.
  • When get powered on, they still get their time from the host. The user is advised to modify the biossystemtimeoffset in Advanced Security Guide, chapter Network Time Synchronization. [20]
  • Time Sanity Check before sdwdate will be executed, this ensures, that the host clock is sane and not slow in 1980. User gets advised to fix its host clock in such cases.
  • After they are connected to the Tor network, they use sdwdate (inspired by tails_htparchive.org) to set the system clock. [21]
  • Time Sanity Check after sdwdate was executed. This should catch eventual bigger bugs and attacks. User gets informed if it fails.
  • Using Boot Clock Randomizationarchive.org, i.e. after boot, the clock is set randomly between 0 and 180 seconds into the past or future. This is useful to enforce the design goal, that the host clock and Whonix-Workstation clock should always slightly differ. It is also useful to obfuscate the clock when sdwdate itself is running, because naturally at this time, sdwdate hasn't finished.
  • sdwdate runs after booting.
  • Whonix-Gateway: Using sdwdate is better against Attack (1) when using a bridge.
  • Every hour, at a random time, sdwdate will set the clock again. This is useful for machines running for long time periods without reboot.
  • sdwdate runs at non-predictable times to prevent the ISP or Tor guard/bridge node to guess, that the user is running Whonix.
  • sdwdate adds or subtracts a random amount of seconds in range of 0.000000001 and 0.999999999. [22] This is supposed to prevent time servers tracking individual users.
  • That will reach the design goal, that all clocks, the host's, Whonix-Gateway and Whonix-Workstation slightly differ.

Block Networking until sdwdate Finishes[edit]

TODO: expand

This is not yet enabled by default.

Network access will be limited to Tor and sdwdate until network time synchronization is done to prevent accidental traffic before that is done.

Whonix firewall runs in two different modes. After boot it runs in timesync-fail-closed mode. 

OK: First run during current boot, therefore running in timesync-fail-closed mode.
OK: (Only local Tor control port connections and torified sdwdate allowed.)

Otherwise after sdwdate succeeded or after manually reloading Whonix firewall it runs in full mode. 

OK: Consecutive run during current boot, therefore running in full mode.
OK: (Full torified network access allowed.)

Users are encouraged to test so we can enable it by default in the future.

https://forums.whonix.org/t/testers-wanted-blocking-networking-until-sdwdate-finished-status-of-sdwdate-gui/5372/6archive.org

Instructions:

Network_Time_Synchronization#Block_Networking_until_sdwdate_Finishes

boot clock randomization[edit]

Introduction[edit]

The TimeSync page notes:

  • Using Boot Clock Randomization, i.e. after boot, the clock is set randomly between 0 and 180 seconds into the past or future. This is useful to enforce the design goal, that the host clock and Whonix clock should always slightly differ. It is also useful to obfuscate the clock when sdwdate itself is running, because naturally at this time, sdwdate hasn't finished.
  • sdwdate runs after booting.

By randomly moving the system clock a few seconds (and nanseconds) in the past or future during boot, this enforces the design goal of a slightly different host clock and VM clock, even before secure time synchornization has succeeded. This prevents time-based fingerprinting and linkability issues, thereby improving anonymity and privacy. [23]

For technical discussion on the Boot Clock Randomization design, see herearchive.org. [24]

Log Inspection[edit]

sudo journalctl -b --no-pager -u bootclockrandomization

Conclusions[edit]

When the user powers on Whonix-Gateway and the host time is too much off, it will not be able to connect to the Tor network. It is advised, when powering on Whonix-Gateway, to check that the host time is no more than 1 hour in past or more than 3 hour in future, otherwise Tor will be unable to establish circuits. [25]

An adversary tampering with the user's clock, while the user doesn't recognize that, can't do any more damage to Whonix than he could do to Tor Browser. Worst case is a denial of service for Tor. On the other hand, an adversary capable of actively tampering with the traffic between the user and its entry guard or bridge poses much bigger risks to Tor in general. [26]

The reason for running TimeSync on Whonix-Gateway, is that onion services can only be accessed if the clock is no more than 30 minutes slow or fast. [27] Running TimeSync ensures, that Whonix-Gateway clock is reasonably accurate.

External Clock[edit]

This topic is still under research. Help is welcome.

It might make sense to add an external clock, such as a GPS, or even better an atomic clock. (Can we get an atomic USB clock?) This clock should be added to the host and/or Whonix-Gateway and/or Whonix-Workstation?

Open question: would the GPS/atomic clock be too accurate and would that make Whonix more fingerprintable?

Discarded Options[edit]

NTP[edit]

Procedures

tlsdate[edit]

https://github.com/ioerror/tlsdatearchive.org

Pros:

  • uses seccomp
  • written by Jacob Appelbaum (security researcher; The Tor Project staff member) Due to the recent news now probably abandoned.

Cons:

  • Only native SSL CA pinning support. No direct SSL certificate pinning support.
  • Does not distribute trust. [31] [32] [33] The single time source can send by error or by malicious intent send wrong time replies.
  • Does not support hooks for user notifications about state of network time synchronization. → Users shouldn't use the internet in Whonix-Workstation before network time synchronization finished, and timesync is informing users about this. → Implementing notifications would be difficult or require patching tlsdate.
  • Produces clock jumps, which confuses various applications. Does not gradually adjust as sdwdate does with Slow Clock Adjusterarchive.org. [34]
  • Cannot connect to (most) Tor onion services, because most of those do not support SSL. [35] [36]
  • Minor: command line parser doesn't fail closed on extraneous / unknown command line parameters [37] - not that important in the absence of bugs, butsafer behavior for tlsdate would be to fail closed on on extraneous / unknown command line parameters.
  • Likely denial of service issue: https://github.com/ioerror/tlsdate/issues/149archive.org

htpdate[edit]

Original.

Tails htpdate fork

  • code in revision control: https://salsa.debian.org/debian/htpdate.gitarchive.org
  • Tails' reasons to fork htpdate:
    • they wanted to add features such as TLS Authentication of servers
    • and source pools model. [38]
  • The htpdate implementation used by Tails is the perl one and not the C version.

Manual Time Setting[edit]

Prompt are very difficult.

  • GUI users:
    • Whonix users: Most laymen don't know what UTC is. Timezone/location should not be asked for privacy reasons too. Users would have to set the time in UTC. At probably a popup would say "use the clock widget and set clock in UTC" which is bad usability.
    • Kicksecure™ users: Prompt could be in the timezone as defined by user during installer. Slightly less worse.
  • CLI users:
    • Servers: no prompt possible for a server that reboots (such as after forced server restart).

TODO[edit]

Consideration no TimeSync on Host[edit]

Would it be advisable to run no network time synching daemon on the host at all? There are open questions. See:
https://forums.whonix.org/t/ntp-possible-skew-per-day-month-year/487/5archive.org

Time Source of Time Servers[edit]

Even if we could very securely obtain the time from a server with distributed trust and everything, the question is, how do these servers themselves set their own clock? If they just use unauthenticated NTP themselves, they might not be a big help.

TODO: invent sdwdate-server

Tor Consensus Method[edit]

Introduction[edit]

Just an idea. Not implemented.

Tails' tordatearchive.org (Guessing time based on Tor consensusarchive.org), renamed to anondate for trademark reasons and refactored to work outside of Tails provides useful bash functions to check system clock using Tor's consensus file and Tor's log.

https://github.com/Whonix/helper-scripts/blob/master/usr/sbin/anondatearchive.org

Tor connects to the network. The Tor consensus file contains rough time information. Since when and until when it is valid.

tordate (now anondate) can provide us with a few machine readable infos we can then use for whatever we may need them.

This is magic. 

valid-after=2014-07-31 21:00:00
valid-until=2014-08-01 00:00:00

This is simply /bin/date. 

Current time is 2014-07-31 22:44:02

Less magic. 

Current time is in valid Tor range

Less magic, but maybe useful. 

Current time is not in valid Tor range, setting to middle of this range: [2014-07-31 22:30:00]

anondate can do this by parsing Tor's verified consensus file /var/lib/cached-microdesc-consensus or by parsing /var/lib/tor/unverified-microdesc-consensus.

Tor Consensus Download and Verification Conditions[edit]

Under which conditions Tor is not able to verify the Tor consensus file?

1) If it has been tampered with by a man-in-the-middle.

2) For clocks that are more than 30 days in the past or 2 days in the future, Tor will not even be able to download a consensus. In this case anondate can parse Tor's log and report the authority's cert's valid-after date. [39] 

anondate --tor-cert-valid-after

Jun 16 00:00:00 2014 GMT
Open Questions[edit]
  • Is the authority's cert's valid-after date signed/verified/built-into-Tor or can it be tampered with by a man-in-the-middle attack?
  • When clock is more than 1 hour in past or more than 3 hour in future, Tor will be unable to establish circuits? [40]
Required Clock Accuracy for Onion Services Accessibility[edit]
Gateway[edit]

Generally, computer running Tor (Whonix-Gateway):

live consensus definition, quote Tor Tor directory protocolarchive.org:

A network-status document is "live" if the time in its valid-after field has passed, and the time in its valid-until field has not passed.

reasonably live consensus definition, quote Tor Tor directory protocol:

A reasonably live consensus is one that expired less than 24 hours ago.

How accurate has the clock to be so onion services can be accessed? [41]

Due to issue Most Onions Down due to a Denial of Service Attack on the Tor Networkarchive.org this question received attention.

Previously a live consensus was required. From Tor 0.4.5.3 and above only a reasonably live consensus will be required.

Workstation[edit]

Generally, computer running applications (such as curl or sdwdate) connection to onions (Whonix-Workstation):

Applications running on a separate computer/VM (Whonix-Workstation) can establish onion connections irrespective of ultra slow or ultra fast system clock on that computer. That is as Tor on a separate computer/VM is functional, which however requires a correct clock as described above.

Summary[edit]
  • Whonix-Gateway: Clock need to be correct enough to be able to reach onions.
  • Whonix-Workstation: No clock correctness requirements for ability of reaching onions.
Reasons for relying solely on anondate for time setting[edit]
  • Does not require [https://github.com/Whonix/Whonix/issues/24 ssl certificate pinning].
  • Does not rely on remote web servers.
  • Does not require a somewhat accurate clock as Remote Web Servers Method requires to get a rough time guess.
anondate issues[edit]

Originally posted in sdwdate Tor Consensus Time Sanity Checkarchive.org:

  • As per chat log with Rogerarchive.org directory authorities can lie about time - so we need to use this with care.
  • Tor consensus isn't always downloaded from directly from directory authorities, sometimes Tor downloads the Tor consensus from directory mirrors. And the latter aren't trusted as much as directory authorities. Those are just like normal relays.
  • We should have the courtesy to not explicitly download from directory authorities, because...

    armadev: oh. i think that would be horrible. hundreds of thousands of users doing that could overwhelm the directory authorities.

Reasons against relying solely on anondate for time setting[edit]
  • Provides only rough time.
    • web fingerprint would differ from most other Tor users.
    • Would cause lots of support requests complaining about skewed clocks.
    • Might trouble onion services (such as hidden web servers) that need to show time to clients?
    • Might trouble people attempting to tunnel I2P through Tor?
    • Might cause other (gpg) verification issues?
      • Debian updater APT refuses to use sources which are not yet valid. I.e. source with a signed valid-until field later than the local system clock.
  • Fingerprinting issues at ISP level:
  • Hidden services can only be accessed if the clock is no more than 30 minutes slow or fast. [42]
  • Tails ticket: Robust time syncingarchive.org Quote:

    Currently we use tordate to set the initial clock. It is very messy and error prone, so we desperately need a robust replacement.

  • Vulnerable to ISP level man-in-the-middle attacks:
    • Tails Design: Time Syncingarchive.org Quote:

      First, replaying a consensus older than one week or so results in preventing access to the Tor network, and that's all, because onion keys will be wrong.

      • In a situation where an adversary is replaying a more than ~7 days old consensus:
        • anondate could report a clock that is more than ~7 days slow. With a system clock that is that slow, Tor would no longer be able to establish circuits. But there are multiple reasons why Tor may not be able to establish circuits. So anondate could only guess, that the clock reported in Tor consensus is more than ~7 days slow, but not know with certainty.
      • In a situation where an adversary is replaying an up to ~7 days old consensus:
        • anondate could report a clock that is up to ~7 days slow and Tor would still be able to create circuits.
        • With Tor as is at the moment, anondate cannot even in principle provide a more accurate time guess than a clock that is up to ~7 days slow.
        • Using an up to ~7 days old consensus, is most likely something we really should avoid?
    • Therefore vulnerable to Attack 3 as per #Attacks.
Conclusions[edit]
  • Relying solely on anondate for time setting is probably a bad idea.
  • Tor consensus time is used as a sdwdate Tor Consensus Time Sanity Check.
  • Rough time fix.
    • If clock is too much off so sdwdate cannot fix it, use anondate to get a rough time fix, then let sdwdate do the rest.
Why not use tordate by Tails instead of reinventing the wheel with anondate?[edit]
  • Because Tails code is highly Tails specific.
    • Depending on non-persistent Tor data dir.
    • Entangled with Tails specific gui notifications.
    • Entangled into the Tails specific boot process with other scripts.
    • Not a clean abstraction of "tell me what Tor's log / Tor's consensus thinks about date/time".
    • Quote: "Currently we use tordate to set the initial clock. It is very messy and error prone, so we desperately need a robust replacement." [43]
Random Stuff[edit]

Tor Certificate Lifetime[edit]

certificate lifetime 

Certificate not yet valid. Either their clock is set wrong, or your clock is wrong.
(certificate lifetime runs from Aug 27 00:00:00 2020 GMT through Aug 27 00:00:00 2021 GMT. Your time is Jan 09 07:33:33 2000 UTC.)

sudo anondate --tor-cert-valid-after 

Aug 27 00:00:00 2020 GMT

Might be related to the following. Quote Tor Tor directory protocolarchive.org:

Every authority has a very-secret, long-term "Authority Identity Key". This is stored encrypted and/or offline, and is used to sign "key certificate" documents. Every key certificate contains a medium-term (3-12 months) "authority signing key", that is used by the authority to sign other directory information. (Note that the authority identity key is distinct from the router identity key that the authority uses in its role as an ordinary router.)

While Authority Identity Key might be hardcoded in Tor, the certificate lifetime is likely not. Otherwise Debian (old)stable would have to ship yearly upgrades to upgrade the "Tor directory authority certificate start date" which never happens. Therefore certificate lifetime should not be trusted since it could be stale.

Tor Certificate Lifetime likely has a digital software signature which gets verified by Tor. It's however not a trustworthy source of time. Vulnerable to replay attacks. Could be one year or older. That's because to verify its freshness, Tor requires a correct clock. For time synchronization purposes however the clock is assumed to be incorrect. What however could be trusted that at least that time has already come. It's not a far future date because then Tor directory authorities wouldn't have signed a Tor certificate with that lifetime yet.

Related source code: https://src-ref.docs.torproject.org/tor/x509__openssl_8c.html#adcb9e22c87cb9ba135f6edf3e13386b4archive.org

Related ticket: https://gitlab.torproject.org/legacy/trac/-/issues/4370archive.org

Tor Consensus Issues[edit]

Quotearchive.org Roger Dingledine:

Whether this is a good idea depends on where you got the consensus. If you connect to a Tor directory mirror and it hands you a consensus from last month, and you set your clock based on it, then you've opened yourself up to exactly the attack that Tor is trying to defend against.

Problem is, nowadays Tor in most cases doesn't download from Tor directory authorities anymore. And we ought to not download the Tor consensus exclusively from Tor directory authorities. Source: [44]

<armadev> oh. i think that would be horrible. hundreds of thousands of users doing that could overwhelm the directory authorities.

https://forums.whonix.org/t/tor-dev-mailinglist-proposal/489/printarchive.org

Fixing Time based on Tor Consensus[edit]

Future Implementation consideration.

This could be how time syncing is performed when Whonix starts:

Credits: This is a rewrite of Tails Time Syncing Designarchive.org

  1. Start Tor. If Tor is already working, skip to the following anondate steps.
  2. Let Tor fetch the Tor consensus.
  3. (anondate related step) If the time is too badly off, the Tor directory authority certificate may not be valid, so we set the system time to the Tor directory authority certificate start date. (sudo anondate --tor-cert-valid-after) For example, see above. Which will guarantee that the Tor directory authority certificate will be valid and the Tor consensus (which includes Tor valid-until date/time) can be downloaded.
  4. (anondate related step) Wait until Tor can tell us the Tor valid-until date/time.
  5. (anondate related step) Set system time to Tor consensus time valid-until - 1:30. (In other words: minus 1 and 30 or minus 90 minutes.) (sudo anondate --show-middle-range) [45]
  6. Wait until the Tor bootstrap process has been completed.
  7. Use sdwdate (which exclusively uses onions as time sources) though Tor to get a more correct system time.

A notification in form of sdwdate-gui shown while the whole process is running.

Non-issues:

  • Use of unverified Tor consensus is not required.
  • The hardware clock is not affected.

Security discussion:

sdwdate's approach essentially removes Tor's time skew check, which is used to prevent replay of consensus data. Let's discuss this class of attacks.

First, replaying a consensus older than four weeks or so results in preventing access to the Tor network, and that's all, because onion keys will be wrong. An attacker who is in a position to replay a consensus to the user could anyway do this, unrelated to time, so the issue at hand boils down to replaying a consensus not older than four weeks or so.

Second, the same type of attacker as above could also try to forge a completely new consensus, which would be unverifiable since the attacker doesn't have access to the authorities' keys. An attacker in that position could do denial-of-service attacks in many other ways, so this doesn't make the situation any worse.

Third, things are different depending on if the user is using a bridge or not.

If not using a bridge, first boot:

  • Tor starts without a cached consensus.
  • The Tor client starts by connecting directly to a fallback directory mirrorarchive.org to download the Tor consensus.
  • Feeding a stale consensus requires the attacker either to break SSL or to control the fallback directory mirror the user's Tor client connects to.
  • Under these conditions the adversary could feed an up to four weeks stale Tor consensus, which is used until sdwdate has fixed the time.
  • Not good, but probably a compromise we can make.

If using a bridge, first boot:

  • Tor starts without a cached consensus.
  • Tor fetches Tor consensus from the bridge.
  • Feeding a stale consensus requires the attacker either to break SSL or to control the user's bridge.
  • Under these conditions the adversary could feed an up to four weeks stale Tor consensus, which is used until sdwdate has fixed the time.
  • Not good, but probably a compromise we can make.

If your ISP, Tor entry guard or Tor bridge, can also:

  • A) set up an onion MitM attack against the sdwdate onion connections, and/or
  • B) controls onions used by sdwdate as time sources,

it can trick you into keep using this old consensus for max. four weeks, which is much worse.

A) should be extremely unlikely. A successful MitM against a Tor onion connection have never been reported. While Tor onions servers might be vulnerable to various de-anonymization attacks, this is a different topic, this affects only the anonymity of the onion server. The client, visitor (here user's machine using sdwdate) does not have to worry about this. Also does not affect the encryption/authentication of the connection. No reports of breaking onion encryption/authentication where known at time of writing. If this was possible, this would be catastrophic anyhow. This wouldn't make the situation any worse.

B) should also be very unlikely. Onion servers targeting specific users seems very unlikely since users connect anonymously using Tor to an onion server. An attack against all users is also very unlikely since sdwdate is distributed (uses median of 3 onion time sources), randomly chosen from sdwdate time sources list and such an attack would be easy to notice, probably reported by users and in response developers would remove these sdwdate time sources.

In any case it should be better than using unauthenticated NTP.

Tor Consensus Time Setting Method Security Considerations[edit]

User local system clock being much skewed is a big issue because:

  • a) Stands out in javascript snooping scripts (Even if Tor Browser fixed that, other applications could leak it.)
  • b) Prevents Tor consensus from being downloaded leading to broken Tor connectivity.
  • c) Other issues discussed on top of this page, Time Attacks.

It depends on the start conditions. Seems really complex. The user's clock is either:

  • [C]: correct
  • [D]: incorrect

The Tor directory mirror (just normal relays) is either sending the user a Tor consensus that is:

  • [E]: fresh
  • [D]: legitimate (bug) stale
  • [F]: malicious (targeted attack) stale
  • [G]: invalid

Now these are quite a few cases to combine and consider. [G] we can ignore. [D] and [F] can be regarded as approximately the same issue.

user clock slow [D] + stale Tor consensus [F]: user would remain on fake Tor network

user clock correct [C] + stale Tor consensus [F]: anondate would introduce a time skew. Not fix one. The user's system clock would be set to the past because the stale Tor consensus told anondate so. The worst could be prevented with a hardcoded minimum date as a safeguard but it's still a security compromise to make.

user clock slow [D] + fresh Tor consensus [E]: In this case anondate would fix the connectivity/usability issue, fix a) and b).

Then it also depends on how slow the user's clock is.

  • [H] Less than 24 hours slow clock or
  • [I] More than 24 hours slow clock.

Therefore:

  • [H] + a): issue
  • [I] + a): no issue since no connection
Issues[edit]

The issue with looking at Tor consensus time is that after power off for a few hours (>= ~3 hours) Tor starts necessarily with a stale Tor consensus. When querying Tor consensus time (using anondate-get or otherwise), from the perspective of anondate-get the clock is fast, because Tor consensus is stale because Tor hasn't downloaded a newer Tor consensus yet.

This happened up to Whonix 16.0.3.1 after the gateway VM was offline for a few hours: 

sudo anondate-get
/usr/sbin/anondate-get: INFO: 100% Tor bootstrap, ok.
/usr/sbin/anondate-get: INFO: tor_circuit_established_check.py, ok.
/usr/sbin/anondate-get: INFO: Tor circuit already established, ok.
/usr/sbin/anondate-get: WARNING: local system time is NOT with valid time range. (valid_after: 2021-05-25 11:00:00 | middle_range: 2021-05-25 12:30:00 | valid_until: 2021-05-25 14:00:00)
/usr/sbin/anondate-get: INFO: middle_range: '2021-05-25 12:30:00'
/usr/sbin/anondate-get: INFO: time_result later than minimum-unixtime-show, ok.
2021-05-25 12:30:00
/usr/sbin/anondate-get: END: Exiting with exit_code '0' indicating 'Showed Tor consensus time middle range.'.

/usr/lib/helper-scripts/onion-time-pre-script: Start.
Static Time Sanity Check: Within minimum time 'Sun Jan 17 00:00:00 UTC 2021' and expiration timestamp 'Tue May 17 10:00:00 UTC 2033', ok.
Tor reports: NOTICE BOOTSTRAP PROGRESS=95 TAG=circuit_create SUMMARY="Establishing a Tor circuit"
Tor circuit: not established.
Tor Consensus Time Sanity Check: The clock might be too fast. Clock is faster than consensus/valid-until 2021-05-25 14:00:00.
/usr/lib/helper-scripts/onion-time-pre-script: END: Exiting with exit_code '2' indicating 'wait, show busy icon and retry.'.

The clock might be too fast. is because while the hardware clock was reasonable correct, Tor consensus time was still stale since Tor didn't download a new Tor consensus yet.

After approximately two minutes after VM start (Tor finished downloading a newer Tor consensus): 

sudo anondate-get
/usr/sbin/anondate-get: INFO: 100% Tor bootstrap, ok.
/usr/sbin/anondate-get: INFO: tor_circuit_established_check.py, ok.
/usr/sbin/anondate-get: INFO: Tor circuit already established, ok.
/usr/sbin/anondate-get: INFO: local system time is already within valid range, ok. (valid_after: 2021-05-25 18:00:00 | middle_range: 2021-05-25 19:30:00 | valid_until: 2021-05-25 21:00:00)
/usr/sbin/anondate-get: INFO: time_result later than minimum-unixtime-show, ok.
/usr/sbin/anondate-get: INFO: Tor certificate lifetime valid, ok.
/usr/sbin/anondate-get: END: Exiting with exit_code '4' indicating 'Setting time using anondate either not possible or not required.'.

Static Time Sanity Check: Within minimum time 'Sun Jan 17 00:00:00 UTC 2021' and expiration timestamp 'Tue May 17 10:00:00 UTC 2033', ok.
Tor reports: NOTICE BOOTSTRAP PROGRESS=100 TAG=done SUMMARY="Done"
Tor circuit: established.
Tor Consensus Time Sanity Check: Clock within consensus parameters consensus/valid-after 2021-05-25 18:00:00 and consensus/valid-until 2021-05-25 21:00:00.
Tor already reports circuit established.
/usr/lib/helper-scripts/onion-time-pre-script: END: Exiting with exit_code '0' indicating 'success'.

Comparison: 

valid_after: 2021-05-25 11:00:00 | middle_range: 2021-05-25 12:30:00 | valid_until: 2021-05-25 14:00:00
valid_after: 2021-05-25 18:00:00 | middle_range: 2021-05-25 19:30:00 | valid_until: 2021-05-25 21:00:00

Actual time when this happened was: 

2021-05-25 18:40:00

If sdwdate had set the time to anondate-get's initial guess 2021-05-25 12:30:00 (using anondate-set), Tor connectivity would have been broken. (This issue was experienced indeed in past during experimentation using anondate-set.)

Tor control protocol has no feature to query "attempting to download a newer Tor consensus done yet?"

Idea:

If hardware clock is faster than valid_until, delete Tor consensus before Tor starts.

(Easily implemented thanks to systemd units.) Tor would re-download the Tor consensus anyhow. But this might be fingerprintable but then again, being a user of Whonix might be fingerprintable anyhow (random ISNarchive.org / security-misc network hardening). Instead of Tor starting with a stale consensus, figuring out it's stale and re-downloading one, it would go straight ahead and download a new one which it has to do anyhow.

Maybe using anondate-set to fix the clock on Whonix-Gateway only. That might mitigate introducing new fingerprinting issues inside Whonix-Workstation. And sdwdate in Whonix-Workstation should be functional anyhow as soon as Tor in Whonix-Gateway is functional.

Maybe most issues with broken hardware clocks are these being too slow, and not too fast? Perhaps by only ever using anondate-set to move the clock into the future but never to the past most if not all security issues can be prevented.

If Tor relays send us a too new Tor consensus, they are in position for denial of service anyhow. Too new is unlikely since for that Tor directory authorities would need to be malicious/compromised. Even if that happens, it would just DOS Tor. Any TLS certificate (debian APT) would be refused if clock is too far in future. Security issues with wrong system time seems to be only with too slow clocks but not with too fast clocks. The latter is at worst a DOS issue.

Clock Fixing[edit]

Since a few Whonix 16 point releases now.

If Tor is running on the same machine / VM (such as Kicksecure or Whonix-Gateway):

  • very slow clocks can be fixed
  • very fast clocks cannot be fixed

If Tor is running on another machine / VM (such as Whonix-Workstation):

  • very slow clocks can be fixed
  • very fast clocks can be fixed

That is because Tor is not running on these machines. Once Tor is functional on Whonix-Gateway, it is easy for sdwdate in Whonix-Workstation to connect to onion time sources despite how far skews the time is.

Notes[edit]

Adjusting time slowly using adjtimex/ntp_adjtime[edit]

Under considerationarchive.org is changing the time with sclockadj2. The initial drive for this was that calling clock_settime() in the original sclockadj can result in several thousand entries in systemd's journal per invocation (can be seen with journalctrl -f) on Debian.

sclockadj2 uses adjtimex call in libc (als called ntp_adjtime) which translates to Linux' kernel system call 124. This is a modal interface that is used by `ntpd` to change the system clock. adjtimex supports a mode with "hard" changes to the time, but those generate entries in systemd journal, just like with clock_settime.

The offset mode for adjtimex allows a change of 0.5 seconds per call (used to be ~130 milisec until the 2.6 kernel). The documentation of this call is incomplete, partly resulting from the limited number of clients (ntpd, chronyd) using this interface, but by tracing the kernel internals (linux/kernel/time/ntp.c and /linux/kernel/time/timekeeping.c) one can get a more complete picture, including that modern kernels support a NANO mode easier implementation of smaller adjustments than the older μsecond based interface, without the need to do clock tick adjustment calculations.

Based on this interface and mode bit definitions taken from /usr/include/linux/timex.h (more complete than the counterparts you typically find online) to form a Python based interface via ctypes. Primarily first setting the kernel internals to handle nanoseconds specifications and run in PLL mode. Then using OFFSET to adjust the time. With this call the kernel is instructed to change the time gradually. You can observe this change by making further calls to adjtimex, this time without setting any values (specified by 0 in the modes bitfield of the structure handed to adjtimex). To handle changes of more than the maximum allowed (±0.5 seconds), you need to invoke a 0.5 second change and observe if the offset has been reduced to 0, then invoke again. Writing a new offset using adjtimex will overwrite any running adjustment process, therefore you have to wait.

By default the kernel uses a decreasing stepsize as the (remaining) offset to handle approaches 0. This can be influenced by the TIMECONST to start with bigger steps, or even reset the stepsize on a regular basis while the kernel is changing the clock according to the remaining offset.

The ntpd can use this offset changing possibility as is but also has a more sophisticated approach, by instructing the kernel about the drift compared to the internal clock.

The sclockadj2 directly uses a nanosecond offset (positive or negative) that instructs the kernel to adjust (looping multiple invocations if the absolute value of the offset > 0.5 seconds. The program has to be run as root in order to tell the kernel to make these changes (querying the status doesn't require this). At the start of this process a PID file is written, which is checked during loops (if running longer than 0.5 s) and will kill any previous invocation of the program. The syntax for the offset invocation is: 

     sclockadj2 offset [-h] [--constant CONSTANT] [--verbose] [--quiet][--wait]  nanoseconds

for changes > 0.5s --wait has to be specified. Increased verbosity can be used to watch the progress of what the kernel is doing with the offset.

This progress can also be seen by invoking the program with the status option. To do so has some instructional value for observing normal `ntpd` behaviour as well. The full options for status: 

   usage: sclockadj2 status [-h] [--follow] [--set] [--verbose] [--quiet]
                            [--debug] [--wait]

   adjtimex status (--quiet → only offset, --verbose → detail)

   optional arguments:
     -h, --help     show this help message and exit
     --follow, -f   follow status indefinitely
     --set          set status explicitly to PLL and NANO
     --verbose, -v  increase verbosity level
     --quiet, -q    decrease verbosity level
     --debug, -D    Debug messages. Don't change date
     --wait         wait for offset to return to 0

Stopping a previous invocation of sclockadj2 (which might be looping on values > 0.5s), including stopping the internal offset can easily be achieved by calling sclockadj2 offset 0

Blockchain Time[edit]

NTS - Network Time Security - Secure NTP[edit]

roughtime[edit]

roughtime information[edit]

roughtime client list[edit]

Debugging[edit]

Useful to run in Qubes dom0. 

qvm-run --pass-io {{project_name_gateway_vm}} "date -u" ; date -u

See Also[edit]

Other[edit]

Previous Discussion[edit]

Other Projects[edit]

Comparison with Others[edit]

Network Time related

Tickets[edit]

Qubes[edit]

high priority[edit]

normal priority[edit]

others[edit]

Clock Correlation Attack[edit]

Attack[edit]

prerequisite knowledge:

  • Whonix-Gateway [Proxy]VMs and Whonix-Workstation [App]VM's system time cannot not be obtained by watching traffic at ISP level.
  • This chapter is not about tracking a single VM across its different Tor exit relays.
  • The context is anonymity. The significance is deanonymization.
  • In anonymity we don't just care about about correlations. Also about estimations. Anonymity set reduction. Partitioning users is already a success. Because it may be combined with other guesses. No direct proof is required. From some countries, there are just a few 100 users or less. If there is already a lead by country, then combined with these time related issues, things begin to really matter.
  • clocksource 'xen' / 'kvmclock' doesn't prevent Whonix from using sdwdate.
  • clocksource 'xen' / 'kvmclock' doesn't prevent each VM having slightly different system time (used by all the applications), so no direct leaks here.
  • [host | dom0] host time and VM times a very, very similar. At most one second difference. Or less. In [host | dom0], see:
date ; qvm-run --pass-io sys-net date

Attack:

  • 1) requires a) compromise a Whonix-Workstation VM and b) observing the ISP network the user is using
  • 2) read [host | dom0] time using clocksource 'xen' / 'kvmclock' or possibly the wall clock interface
  • 3) [host | dom0] time will be very similar to NetVM's time (the system where external network connections are made)
  • 4) correlate Whonix-Workstation VM with NetVM's time
  • 5) anonymity set reduction done

Notes:

No need to compromise multiple VMs. Only one VM, a Whonix-Workstation VM needs to be compromised. NetVM does not have to be compromised, it leaks in most cases - hard to avoid all cases - any passive ISP level adversary watching traffic can read the clients local clock. It is leaked on many levels, see #Local Clock Leaks.

Threat Model:

  • Whonix-Workstation AppVM gets compromised up to local root code execution. Then the attacker uses the local exploit, reads clocksource 'xen' / 'kvmclock'.

Defense[edit]

We need to make sure that all of Whonix-Gateway, Whonix-Workstation, and NetVM uses slightly different time. Even if Whonix-Workstation would be compromised, and the attacker gets knowledge of the time of [host | dom0] and sniffs NetVM's time, it would hinder correlation.

[host | dom0] and non-anonymous VM's clocks should be as synced, correct and secure as those can be.

But those should differ from the clocks within Whonix VMs.

We need completely isolated virtual time within Whonix VMs. [46] [47]

wallclock[edit]

> The other interface ("wallclock interface") would provide such time, though. And it is not configurable. But not sure if it is used anywhere in Linux by default (probably besides initial system time set).

Yes. Must be so. Otherwise with "ticks" alone the VM had no chance to figure out the current time. And this is a roblem, because we don't want the VM to know [host | dom0]'s time.

We might be able to solve the wallclock [not pvclock] issue by Spoofing the Initial Virtual Hardware Clock Offset.

Development[edit]

Rather than modifying [host | dom0]'s clock, we could perhaps spoof the initial virtual hardware clock offset. It is possible for KVM, as per: https://libvirt.org/formatdomain.html#time-keepingarchive.org

Something like this: 

<clock offset='variable' adjustment='123456' basis='utc'>

Reading clocksource xen as a [root] user is not easily possible. source: https://lists.xen.org/archives/html/xen-users/2015-08/msg00020.htmlarchive.org

Would probably require writing a kernel module.

Sources[edit]

TSC[edit]

Needs more research.

Why we should avoid leaking TSC from the host go the VM.

Leaking anything related to time from the host into the VM [or from VM to VM] risks cryptographic side channel attacks and should therefore be avoided.

Search term:
time stamp counter tsc side channel

Leads to:
https://blog.cr0.org/2009/05/time-stamp-counter-disabling-oddities.htmlarchive.org

TSC leaks may be similar to TCP sequence numbers. [48]

clocksource xen[edit]

https://github.com/QubesOS/qubes-issues/issues/1764#issuecomment-195619793archive.org

Resources[edit]

Footnotes[edit]

  1. ↑
  2. ↑ 2.0 2.1 Source: RFC5905archive.org

    Origin Timestamp (org): Time at the client when the request departed for the server, in NTP timestamp format.

  3. ↑ 3.0 3.1 Credits: Thanks to dfcarchive.org for the answer on security.searchive.org.
  4. ↑ By distribution defaults.
  5. ↑ Was fixedarchive.org in Tor Browser, but remains open for other applications such as browsers like Mozilla Firefox.
  6. ↑ See https://ip-check.info/archive.org for demonstration.
  7. ↑ As of June 2016, Iceweasel has been replaced by Firefox-ESR in Debian, see https://wiki.debian.org/Iceweaselarchive.org
  8. ↑ prevent leak via Date header field (local timestamp disclosure)archive.org
  9. ↑ prevent leak via Message-ID header field (local timestamp disclosure) archive.org
  10. ↑ Quote, on October 24th, 2014 sukhbir saidarchive.org:
    "The timestamp disclosures are via the date and the message-ID headers (relevant tickets: #6314archive.org, #6315archive.org). We have submitted patches to Mozilla (902573archive.org, 902580archive.org) that have not yet been merged. The patches probably need more work and we are looking for volunteers to work on them and help us to get them merged. We will be doing a blog post about this soon and put out a call for help."
  11. ↑ 11.0 11.1 For example the one shipped by Debian wheezy.
  12. ↑ https://gitlab.torproject.org/legacy/trac/-/issues/8751archive.org
  13. ↑ See torproject.org #4852: Clients send NETINFO with timearchive.org. The ticket says fixed. However, Debian wheezy (which Whonix 10 was based on) was using Tor 0.2.3archive.org but Nick's patch appeared first in 0.2.4archive.org. It was solved in Whonix 11, because it was based on Debian jessie, that contained a more recent version of Torarchive.org.
  14. ↑ For demonstration, you could use. 
    curl -silent --head torproject.org | grep -i date:
  15. ↑ Replay old Tor consensus, see Tails: Time syncingarchive.org for detailed explanation.
  16. ↑ Cryptographic verification depends on system clock, i.e. a clock two years in past will accept certificates/updates, which have been expired/revoked for two years.
  17. ↑ #Local Clock Leaks are an issue. Imagine the user connects to an adversary controlled web-server with Mozilla Firefox and connects to another web-server controlled by the same adversary with Thunderbird. The adversary can link the anonymous and non-anonymous session, thus deanonymizing.
  18. ↑ If the clock is too much off, it is also easy for an adversary's web-server to detect "Oh, that's the Tor Browser user who's clock is X in past/future.", thus allowing the adversary to link all sessions to the same pseudonym. See Tor Browser upstream bug #3059: Find some way to deal with time-based fingerprintsarchive.org.
  19. ↑ For onion services a correct clock is crucial, see:
  20. ↑ Developer information: If we needed or wanted to render the hardware clock unusable, we could set VirtualBox --biossystemtimeoffset several decades in past or future.
  21. ↑ Stream isolation has been added, for Whonix sdwdate implementation i.e. people using Tor Browser prevent notifying their exit relay, that they are sdwdate or Whonix users.
  22. ↑ Check. 
    grep -i "Made up" /var/log/sdwdate.log
  23. ↑ https://github.com/Whonix/bootclockrandomizationarchive.org
  24. ↑ Notably, one recent change is the 0-5 second time window is no longer excluded in the process as it was found to aid fingerprinting.
  25. ↑ https://lists.torproject.org/pipermail/tor-talk/2012-February/023264.htmlarchive.org
  26. ↑ General Tor problems, Tor is known to be broken against many active attacks. This can not be solved by Whonix. When an adversary is capable of running active attacks, tampering with the time leading into a denial of service is the least of the worries. The adversary could also disrupt the service easier. And as for active attacking in general, there are other attacks which are easier to deploy and which pose a greater danger. Not a Whonix specific problem.
  27. ↑ https://gitlab.torproject.org/legacy/trac/-/issues/3582#comment:3archive.org
  28. ↑ https://web.archive.org/web/20210618153528/http://lists.ntp.org/pipermail/questions/2010-November/028033.htmlarchive.org
  29. ↑ https://gitlab.tails.boum.org/tails/tails/-/issues/6113archive.org
  30. ↑ 60 s * 60 m * 24 h
  31. ↑ As sdwdate does.
  32. ↑ https://github.com/ioerror/tlsdate/issues/112archive.org
  33. ↑ https://github.com/ioerror/tlsdate/issues/143#issuecomment-57053323archive.org
  34. ↑ sclockadjarchive.org
  35. ↑ Because in most cases, there is little need to combine Tor onion services with SSL. connections to Tor onion services are Tor-to-Tor encrypted by default ("not exactly end-to-end").
  36. ↑ https://github.com/ioerror/tlsdate/issues/157archive.org
  37. ↑ https://github.com/ioerror/tlsdate/issues/158archive.org
  38. ↑ https://tails.boum.org/contribute/design/Time_syncing/#index4h1archive.org
  39. ↑ Source, source code comment in tordate: https://gitlab.tails.boum.org/tails/tails/-/blob/master/config/chroot_local-includes/etc/NetworkManager/dispatcher.d/20-time.sharchive.org 
    	# Since Tor 0.2.3.x Tor doesn't download a consensus for
	# clocks that are more than 30 days in the past or 2 days in
	# the future.  For such clock skews we set the time to the
	# authority's cert's valid-after date.
  40. ↑ https://lists.torproject.org/pipermail/tor-talk/2012-February/023264.htmlarchive.org
  41. ↑ https://gitlab.torproject.org/legacy/trac/-/issues/3460archive.org
  42. ↑ https://gitlab.torproject.org/legacy/trac/-/issues/3582#comment:3archive.org
  43. ↑ https://gitlab.tails.boum.org/tails/tails/-/issues/5774archive.org
  44. ↑ https://forums.whonix.org/t/tor-dev-mailinglist-proposal/489/10archive.org
  45. ↑ anondate --show-valid-after 
    2021-01-09 13:00:00

    sudo anondate --show-valid-until 

    2021-01-09 16:00:00

    anondate --show-middle-range 

    2021-01-09 14:30:00
  46. ↑ https://security.stackexchange.com/questions/106406/how-to-get-a-completely-isolated-virtual-time-using-xenarchive.org
  47. ↑ https://lists.xen.org/archives/html/xen-users/2015-11/msg00114.htmlarchive.org
  48. ↑ https://gitlab.torproject.org/legacy/trac/-/issues/16659#comment:10archive.org

License[edit]

Whonix Dev/TimeSync wiki page Copyright (C) Amnesia <amnesia at boum dot org>
Whonix Dev/TimeSync wiki page Copyright (C) 2012 - 2024 ENCRYPTED SUPPORT LP <

This program comes with ABSOLUTELY NO WARRANTY; for details see the wiki source code.
This is free software, and you are welcome to redistribute it under certain conditions; see the wiki source code for details.

Your support makes
all the difference!

We believe security software like Whonix needs to remain open source and independent. Would you help sustain and grow the project? Learn more about our 12 year success story and maybe DONATE!

Retrieved from "https://www.whonix.org/w/index.php?title=Dev/TimeSync&oldid=101436"