| 1 | # Receiving scan results |
| 2 | |
| 3 | Your app accepted an upload and needs to know whether the file is safe before |
| 4 | it serves it. Two endpoints answer that, and a callback can save you the poll. |
| 5 | |
| 6 | This assumes your app's store already has virus scanning granted. If it does |
| 7 | not, see `carlos store scan` in [the CLI page](/docs/cli#carlos-store-scan). |
| 8 | |
| 9 | ## Asking for a scan |
| 10 | |
| 11 | Both endpoints take a Service-role credential and reach only your own app. |
| 12 | |
| 13 | ```http |
| 14 | POST /api/cli/apps/{account}/{app}/store/scan/object |
| 15 | {"key": "uploads/x.pdf", "callback_path": "/hooks/scan"} |
| 16 | ``` |
| 17 | |
| 18 | `callback_path` is optional. You get back `202` and `{"state":"pending"}`. The |
| 19 | scan is queued, so there is no verdict in that response. |
| 20 | |
| 21 | ```http |
| 22 | GET /api/cli/apps/{account}/{app}/store/scan/object?key=uploads/x.pdf&wait=15s |
| 23 | ``` |
| 24 | |
| 25 | This reads what is known about one object. `wait` long-polls: the request holds |
| 26 | open until the state changes or `wait` elapses, capped at 30 seconds. Ask for |
| 27 | longer and you get 30 seconds. Leave `wait` off and you get an immediate answer. |
| 28 | |
| 29 | ## The states |
| 30 | |
| 31 | ```text |
| 32 | {"state":"clean"} |
| 33 | {"state":"infected"} |
| 34 | {"state":"skipped"} |
| 35 | {"state":"failed"} |
| 36 | {"state":"denied"} |
| 37 | {"state":"pending"} |
| 38 | {"state":"gone"} |
| 39 | {"state":"unavailable","reason":"...","error":"..."} |
| 40 | ``` |
| 41 | |
| 42 | Only `clean` and `infected` are verdicts on the contents. The rest mean |
| 43 | something else: |
| 44 | |
| 45 | - `skipped`: the scanner did not judge the object’s contents, for example because the object exceeds the scan size limit. |
| 46 | - `failed`: the engine read the file and could not reach a verdict. |
| 47 | - `denied`: the console could not read the object. |
| 48 | - `pending`: no answer yet. Ask again, or wait longer. |
| 49 | - `gone`: there is no object at this key. |
| 50 | - `unavailable`: scanning is not available for this store right now. `reason` |
| 51 | is one of `not_granted`, `not_requested`, `encrypted`, `not_enabled`, |
| 52 | `disabled`, `no_engine`, `callbacks_not_configured`. `error` is a sentence |
| 53 | for whoever reads the log. |
| 54 | |
| 55 | If your app treats `skipped` or `gone` as safe to serve, it is claiming |
| 56 | coverage it does not have. |
| 57 | |
| 58 | ## Callbacks take a path |
| 59 | |
| 60 | `callback_path` is a path on your app's ready, platform-assigned host. This is usually `<app>.<account>.<apps-domain>`, but an app may already have a different platform-assigned host. The console uses that route; it does not send callbacks to a custom domain. |
| 61 | |
| 62 | The rules are checked when you enqueue, so a bad value comes back as a `400` |
| 63 | and is never stored: one leading slash, no `//`, no backslash, no `#`, no `..` |
| 64 | segment, no `;`, no `@`, printable ASCII only, and 1KB at most. A query string |
| 65 | is fine and is preserved. |
| 66 | |
| 67 | A request with `callback_path` is refused with HTTP 409 when the app has no ready platform host. Omit `callback_path` and poll scan status, or wait for a ready host and retry. |
| 68 | |
| 69 | ## Verifying the signature |
| 70 | |
| 71 | Every callback carries a header: |
| 72 | |
| 73 | ```http |
| 74 | Carlos-Signature: t=1800000000,v1=ccd4f98ffb69f5dd:1c24c96520fc34a1... |
| 75 | ``` |
| 76 | |
| 77 | `t` is the send time in unix seconds. Each `v1=` field is |
| 78 | `<key-id>:<hex-mac>`, and there can be two of them during a key rotation, one |
| 79 | under the old key and one under the new. Verify against each and accept the |
| 80 | callback if any checks out. Ignore key ids you do not recognise, and ignore |
| 81 | fields you do not recognise, because the platform may add more later and your |
| 82 | verifier should survive that. |
| 83 | |
| 84 | The MAC is HMAC-SHA256 over `t`, then a literal `.`, then the raw request body. |
| 85 | Verify the bytes as they arrived. If you parse the JSON and re-serialise it |
| 86 | first you get a different byte string, and a genuine callback will not verify. |
| 87 | Compare in constant time, and reject the callback if `t` is more than five |
| 88 | minutes from your own clock in either direction. |
| 89 | |
| 90 | ### The hex trap |
| 91 | |
| 92 | This one is boring but read it before you write the verifier. |
| 93 | |
| 94 | Your app receives its callback secret in an environment variable named |
| 95 | `<YOUR_PREFIX>_SCAN_CALLBACK_SECRET`. That value is hex-encoded text. The |
| 96 | signing key is the 32 raw bytes it decodes to, not the string itself. |
| 97 | |
| 98 | Hand the environment variable straight to your HMAC function and every callback |
| 99 | fails to verify, which looks exactly like a platform fault. It is usually this |
| 100 | one line: |
| 101 | |
| 102 | ```python |
| 103 | key = bytes.fromhex(secret_env_var) # not secret_env_var.encode() |
| 104 | ``` |
| 105 | |
| 106 | ### Go |
| 107 | |
| 108 | ```go |
| 109 | func verify(secretHex string, header string, body []byte) error { |
| 110 | key, err := hex.DecodeString(secretHex) |
| 111 | if err != nil { |
| 112 | return fmt.Errorf("callback secret is not valid hex: %w", err) |
| 113 | } |
| 114 | |
| 115 | var timestamp string |
| 116 | var macs [][]byte |
| 117 | for _, field := range strings.Split(header, ",") { |
| 118 | switch { |
| 119 | case strings.HasPrefix(field, "t="): |
| 120 | timestamp = field[2:] |
| 121 | case strings.HasPrefix(field, "v1="): |
| 122 | parts := strings.SplitN(field[3:], ":", 2) |
| 123 | if len(parts) != 2 { |
| 124 | continue |
| 125 | } |
| 126 | mac, err := hex.DecodeString(parts[1]) |
| 127 | if err == nil { |
| 128 | macs = append(macs, mac) |
| 129 | } |
| 130 | } |
| 131 | } |
| 132 | if timestamp == "" || len(macs) == 0 { |
| 133 | return errors.New("missing t or v1") |
| 134 | } |
| 135 | |
| 136 | unix, err := strconv.ParseInt(timestamp, 10, 64) |
| 137 | if err != nil { |
| 138 | return errors.New("bad timestamp") |
| 139 | } |
| 140 | if age := time.Since(time.Unix(unix, 0)); age < -5*time.Minute || age > 5*time.Minute { |
| 141 | return errors.New("timestamp outside five-minute window") |
| 142 | } |
| 143 | |
| 144 | h := hmac.New(sha256.New, key) |
| 145 | h.Write([]byte(timestamp)) |
| 146 | h.Write([]byte{'.'}) |
| 147 | h.Write(body) |
| 148 | want := h.Sum(nil) |
| 149 | |
| 150 | for _, got := range macs { |
| 151 | if hmac.Equal(got, want) { |
| 152 | return nil |
| 153 | } |
| 154 | } |
| 155 | return errors.New("signature does not verify") |
| 156 | } |
| 157 | ``` |
| 158 | |
| 159 | ### Python |
| 160 | |
| 161 | ```python |
| 162 | import hashlib |
| 163 | import hmac |
| 164 | import time |
| 165 | |
| 166 | def verify(secret_hex: str, header: str, body: bytes) -> bool: |
| 167 | key = bytes.fromhex(secret_hex) # the hex trap: decode before using |
| 168 | |
| 169 | timestamp = None |
| 170 | macs = [] |
| 171 | for field in header.split(","): |
| 172 | if field.startswith("t="): |
| 173 | timestamp = field[2:] |
| 174 | elif field.startswith("v1="): |
| 175 | _, _, mac_hex = field.partition(":") |
| 176 | try: |
| 177 | macs.append(bytes.fromhex(mac_hex)) |
| 178 | except ValueError: |
| 179 | continue |
| 180 | |
| 181 | if timestamp is None or not macs: |
| 182 | return False |
| 183 | if abs(time.time() - int(timestamp)) > 300: |
| 184 | return False |
| 185 | |
| 186 | want = hmac.new(key, timestamp.encode() + b"." + body, hashlib.sha256).digest() |
| 187 | return any(hmac.compare_digest(mac, want) for mac in macs) |
| 188 | ``` |
| 189 | |
| 190 | In both examples, `body` is the raw bytes read off the request before any JSON |
| 191 | parsing touches them. |
| 192 | |
| 193 | ## The callback body |
| 194 | |
| 195 | ```json |
| 196 | {"account":"bab","app":"titogo","key":"uploads/x.pdf","etag":"...", |
| 197 | "verdict":"clean","recorded":true,"engine":"clamav","signature":"..."} |
| 198 | ``` |
| 199 | |
| 200 | Match a callback to your own upload on `etag`, the bare ETag your `PUT` |
| 201 | returned, and not on the key alone: the same key can be overwritten. There is |
| 202 | no version token in the body. The platform's internal version format is not |
| 203 | something your app should compare against, and getting that comparison wrong is |
| 204 | a security bug. |
| 205 | |
| 206 | ### Provenance |
| 207 | |
| 208 | `signature` is the scanner's signature-database version. It is separate from the request signature in the `Carlos-Signature` header. `engine` and `signature` can be empty when a verdict came from an existing object tag: the platform does not know which engine database produced it. |
| 209 | |
| 210 | Watch `recorded`. When it is `false` the verdict could not be written onto the |
| 211 | object itself, so an `infected` object with `recorded: false` is not covered by |
| 212 | the platform's enforcement deny. Nothing is blocking reads of that file. Your |
| 213 | app decides what to do about it. |
| 214 | |
| 215 | ## Delivery |
| 216 | |
| 217 | Callback delivery is best effort. A restart or a full delivery queue can lose a notification, and retries can produce duplicates. Make your handler idempotent on `(account, app, key, etag, verdict)`. If no callback arrives within your timeout, poll the `GET` endpoint. |
| 218 | |
| 219 | `GET` can answer `pending`. A verdict that could not be tagged onto the object is kept separately and expires, so polling does not guarantee a retained answer indefinitely. |
| 220 | |
| 221 | The console POSTs with `Content-Type: application/json`, and any `2xx` counts |
| 222 | as delivered. A redirect is a refusal and is not retried, so answer at |
| 223 | `/hooks/scan` directly instead of redirecting to `/hooks/scan/`. A `4xx` is not |
| 224 | retried either. A `5xx`, a `429` or a timeout is retried three times, at 1 |
| 225 | second, 4 seconds and 16 seconds. The console reads at most 4KB of your |
| 226 | response and ignores it. |
| 227 | |
| 228 | ## Rate limits |
| 229 | |
| 230 | Both endpoints share your app's write budget, about 120 requests a minute. A |
| 231 | long-poll `GET` costs two of those a minute for as long as it is open. Polling |
| 232 | in a tight loop with no `wait` will meet the limit, so use `wait` and keep |
| 233 | polling as the backstop. |
| 234 | |
amadan