In standard web architecture, a 500ms latency spike is an annoyance. In low-latency trading, it is a bankruptcy risk.<\/p>\n\n\n\n
When you are competing in microseconds, averages are lies<\/strong>. If your average latency is 10\u00b5s (microseconds), but your 99th percentile is 5ms, your strategy is already dead. You just don’t know it yet because your dashboard is smoothing out the “micro-bursts” that actually kill you.<\/p>\n\n\n\n Most observability tools are built for web servers, not ticker plants. They poll too slowly. If you poll every 10 seconds, you miss the crash that happened at second 3 and recovered at second 4.<\/p>\n\n\n\n Below is the monitoring structure I mandate for any high-frequency system. We break it down into three domains: The Eyes<\/strong> (Market Data), The Hands<\/strong> (Execution), and The Brain<\/strong> (Post-Trade).<\/p>\n\n\n\n <\/p>\n\n\n\n If your system sees the price of Apple (AAPL) as $150.00, but the exchange sees it as $150.05, you are about to sell an asset for less than it is worth. This is “stale data,” and it is the silent killer of algorithms.<\/p>\n\n\n\n The Rules:<\/strong><\/p>\n\n\n\n Once you decide to trade, how fast can you pull the trigger? This measures the “Tick-to-Trade” loop.<\/p>\n\n\n\n The Rules:<\/strong><\/p>\n\n\n\n This is the “sanity check” layer. It ensures that what your algorithm thinks<\/em> it owns matches what the exchange says<\/em> it owns.<\/p>\n\n\n\n The Rules:<\/strong><\/p>\n\n\n\n In low latency, the hardware is<\/em> the software. You cannot ignore the physical layer.<\/p>\n\n\n\n The most important monitoring rule in trading is not a warning; it is an action.<\/p>\n\n\n\n Every metric above should feed into a unified Kill Switch<\/strong>. If the data is too stale, if the rejects are too high, or if the position break is real, the monitoring system must have the authority to pull the plug automatically.<\/p>\n\n\n\n In high-frequency trading, it is better to be offline than to be wrong.<\/p>\n","protected":false},"excerpt":{"rendered":" In standard web architecture, a 500ms latency spike is an annoyance. In low-latency trading, it is a bankruptcy risk. When you are competing in microseconds, averages are lies. If your average latency is 10\u00b5s (microseconds), but your 99th percentile is 5ms, your strategy is already dead. You just don’t know it yet because your dashboard is […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25,18],"tags":[21,14,13,15,28,27,26,30,29],"class_list":["post-52","post","type-post","status-publish","format-standard","hentry","category-observability","category-system-architecture","tag-apm","tag-architecture","tag-product-management","tag-sre","tag-alerting","tag-hft","tag-low-latency","tag-market-data","tag-monitoring"],"_links":{"self":[{"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/posts\/52","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=52"}],"version-history":[{"count":2,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/posts\/52\/revisions"}],"predecessor-version":[{"id":54,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=\/wp\/v2\/posts\/52\/revisions\/54"}],"wp:attachment":[{"href":"https:\/\/platformsignals.dev\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=52"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=52"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/platformsignals.dev\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=52"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}The Eyes: Market Data Feeds<\/h3>\n\n\n\n
\n
\n
# Alert if we are lagging behind the exchange clock latency_skew = local_receipt_time - exchange_packet_timestamp if latency_skew > 50_microseconds: trigger_alert(\"WARNING: Ticker Plant Lagging\") elif latency_skew > 200_microseconds: trigger_circuit_breaker(\"CRITICAL: Stale Prices - HALT TRADING\")<\/code><\/li>\n<\/ul>\n<\/li>\n\n\n\n\n
IF (Current_Seq_Num != Last_Seq_Num + 1) -> TRIGGER_RECOVERY<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\nThe Hands: Order Execution<\/h3>\n\n\n\n
\n
\n
# Measure time from data arrival to order egress processing_time = order_sent_timestamp - tick_arrival_timestamp # We don't care about averages. We care about outliers. if processing_time > (baseline_latency + 3 * standard_deviation): log_warn(\"Micro-burst detected in Strategy Engine B\")<\/code><\/li>\n<\/ul>\n<\/li>\n\n\n\n\n
\u00a0NewOrderSingle\u00a0and receiving\u00a0ExecutionReport (Ack).<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\nThe Brain: Post-Trade & Reconciliation<\/h3>\n\n\n\n
\n
\n
-- Pseudo-query for real-time reconciliation SELECT * FROM internal_trades FULL OUTER JOIN drop_copy_trades ON internal_trades.order_id = drop_copy_trades.order_id WHERE internal_trades.id IS NULL OR drop_copy_trades.id IS NULL<\/code><\/li>\n\n\n\n\n
IF (Rejected_Orders \/ Total_Orders) > 5% within 10s -> KILL_SWITCH_ENABLE<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\nThe Hardware Heartbeat<\/h3>\n\n\n\n
\n
IF rx_discards > 0<\/code>\u00a0on your Solarflare\/Mellanox cards, your CPU is too slow to handle the incoming packet rate. You are flying blind.<\/li>\n\n\n\nIF (Max_Latency - Min_Latency) > 10\u00b5s<\/code>. This usually means “noisy neighbors” on your server or improper CPU isolation.<\/li>\n<\/ul>\n\n\n\nConclusion: The Kill Switch<\/h3>\n\n\n\n