How Many Tokens Is an Image? The Resolution Policy That Decides Your VLM Bill

One 1024×1024 screenshot. Seven vision-language systems. The token count ranges from 576 to 2,880, a factor of five, and the systems at the cheap end physically cannot read the eight-point text in the screenshot's footer because it was destroyed by a bilinear resize before the model saw anything.

That preprocessing step is not a detail. In a modern VLM the language model is a commodity, the vision encoder is nearly a commodity, and the thing that differs most between two systems answering the same question about the same image is the resolution policy: the rule mapping pixels to visual tokens. It sets the floor on what is legible, it sets the prefill cost, and in production it is the single largest line item on a multimodal bill. It is also the part of the stack most teams never configure, because it is buried in a preprocessor and exposed, if at all, as one enum.

Why this matters: Visual tokens dominate the context of any document, screenshot, or video workload. Choosing a resolution policy is choosing an accuracy ceiling and a cost curve at the same time, and the two move in opposite directions. Anyone deploying a VLM without knowing their per-image token count is flying an aircraft without a fuel gauge.

TL;DR

• Visual sequence length scales with image area, and language-model prefill attention is quadratic in sequence length, so doubling each side of an image costs roughly 4× the visual tokens and about 16× the prefill attention work.
• The same 1024×1024 image is 576 tokens to LLaVA-1.5, 630 to GPT-5, 1,032 to Gemini, roughly 1,296 to Qwen2-VL, 1,280 to InternVL 1.5, and 2,880 to LLaVA-NeXT with a 2×2 AnyRes grid. Only some of those can read small text.
• Tiling was the cheap fix and it introduced a new failure: the encoder has no attention across tile boundaries, so any content straddling a seam has to be reassembled by the language model, which is where long numbers and multi-column tables go wrong.
• Visual tokens are attended to far less in deep layers than their count suggests. FastV prunes them after layer 2 and reports large FLOP reductions at modest quality cost, which means much of the resolution you paid for was already redundant before reasoning began.
• Video is where the arithmetic becomes brutal: at Gemini's 258 tokens per tile, one minute sampled at one frame per second of 1024×1024 video is on the order of 62,000 tokens.
• Native-resolution encoders remove the distortion but hand your cost function to the user, since anyone who uploads a 4K image chooses your bill for you. Every production system therefore caps tiles, which quietly reintroduces the downsampling problem for exactly the documents that needed the resolution.

At a Glance

flowchart LR
    A[Raw image] --> B[Resolution policy]
    B --> C[Vision encoder]
    C --> D[Projector or resampler]
    D --> E[LLM prefill]
    E --> F[Answer]
    B -. sets legibility floor .-> F
    B -. sets token count .-> E

    classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff
    classDef amber fill:#b45309,stroke:#fbbf24,stroke-width:1px,color:#fff
    class A blue
    class B amber
    class C,D purple
    class E purple
    class F teal

Everything downstream of the amber box is constrained by it. The encoder cannot recover detail the policy threw away, and the language model cannot afford tokens the policy chose to create.

Before Anyone Thought About Resolution

Vision Transformers arrived carrying an assumption from image classification: images are square, small, and fixed. ViT cut a 224×224 image into 16×16 patches and got 196 tokens (Dosovitskiy et al., ICLR 2021, arXiv:2010.11929). For deciding whether a photograph contains a dog, 224 pixels is plenty. Nobody was reading spreadsheets.

CLIP inherited the constraint and made it universal, because once a contrastively trained encoder at a fixed resolution became the standard visual frontend, every downstream system adopted its preprocessing (Radford et al., 2021, arXiv:2103.00020). SigLIP later replaced the softmax contrastive loss with a pairwise sigmoid so batch size and loss decouple, reaching 84.5% ImageNet zero-shot and finding 32k a sweet spot for batch size rather than the million-scale batches people assumed were needed (Zhai et al., ICCV 2023, arXiv:2303.15343). Still fixed resolution.

The early vision-language models therefore spent their design effort on the opposite problem: images produced too many tokens for the language model to accept. Flamingo interleaved images and text into a frozen language model and used a Perceiver Resampler to squeeze a variable-size grid of encoder features into a small fixed set of latents (Alayrac et al., NeurIPS 2022, arXiv:2204.14198). BLIP-2 did the same job with a Querying Transformer, a small set of learned queries that cross-attend to the frozen encoder output, and beat Flamingo80B by 8.7 points on zero-shot VQAv2 with 54× fewer trainable parameters (Li et al., ICML 2023, arXiv:2301.12597).

Then LLaVA showed that you barely need any of that machinery: a linear projection from CLIP features into the language model's embedding space, trained on GPT-4-generated instruction data, was enough (Liu et al., NeurIPS 2023, arXiv:2304.08485). LLaVA-1.5 refined it to CLIP ViT-L/336px plus an MLP projector, trained on 1.2M public samples in about a day on one 8×A100 node (Liu et al., 2023, arXiv:2310.03744). Simplicity won, and with it came a hard 576-token, 336-pixel budget that the entire open ecosystem adopted.

Which is when people started pointing cameras at documents, and the resize became the whole problem.

timeline
    title From fixed squares to native resolution
    2020 : ViT treats an image as 196 patches at 224 pixels
         : Resolution is a classification-era constant nobody questions
    2021 : CLIP standardises a fixed-resolution frozen visual frontend
    2022 : Flamingo adds a Perceiver Resampler to compress features
    2023 : BLIP-2 Q-Former does it with 54x fewer trainable parameters
         : LLaVA-1.5 fixes 576 tokens at 336 pixels and the field copies it
         : NaViT packs native aspect ratios into one training sequence
    2024 : LLaVA-NeXT AnyRes tiles up to 672x672 and 1344x336
         : InternVL 1.5 scales to 40 tiles and 4K input
         : Qwen2-VL emits variable tokens with M-RoPE
         : Pixtral trains an encoder from scratch for native resolution
    2025 : Serving APIs expose resolution as a billable dial

How a Resolution Policy Actually Works

Three families exist. They differ in what they refuse to give up.

Fixed resize: the baseline everyone still ships

Squash to \(S \times S\), cut into \(p \times p\) patches, emit

tokens. CLIP ViT-L/14 at \(S = 336\) gives \(24 \times 24 = 576\). The cost is constant and known in advance, which is genuinely valuable operationally, and every non-square image is distorted. A 16:9 screenshot squeezed into a square has its text compressed horizontally by 44%, on top of the resolution loss.

The scaling behaviour is what kills it. Visual tokens grow with area, and the language model's prefill attention is quadratic in total sequence length, so the compute to reason over the image grows as the fourth power of the linear resolution:

Going from 336 to 672 pixels is 4× the tokens and about 16× the attention work in the decoder's prefill. That single relation is why "just use higher resolution" was never the obvious answer it appears to be.

Tiling: buy resolution, keep the encoder

Cut the image into crops the pretrained encoder already handles, encode each independently, and prepend a downsampled thumbnail so global layout survives. LLaVA-NeXT selects from a grid set of {2×2, 1×{2,3,4}, {2,3,4}×1}, covering 672×672, 336×1344 and 1344×336 (LLaVA-NeXT, January 2024). With a 2×2 grid that is four crops plus one thumbnail, five encoder passes, \(5 \times 576 = 2{,}880\) tokens.

The virtue of tiling is that it requires no change to the encoder, so it can be bolted onto any CLIP-based model. InternVL 1.5 pushes it hardest: 1 to 12 tiles of 448×448 during training, zero-shot scaled to 40 tiles at test time, which covers 4K input, with a pixel-shuffle that folds each tile's 1,024 patches down to 256 tokens (Chen et al., 2024, arXiv:2404.16821). That 4× compression is what makes 40 tiles affordable at all: without it, 4K input would be over 40,000 visual tokens.

Native resolution: change the encoder

The third family stops fighting and trains the encoder for variable input. NaViT packs patch sequences from several images into one training sequence with a block-diagonal attention mask so images cannot attend across each other, the same sequence-packing trick used in LLM pretraining, and reports improved training efficiency for large-scale supervised and contrastive pretraining plus a smooth test-time cost-performance trade (Dehghani et al., NeurIPS 2023, arXiv:2307.06304).

Qwen2-VL takes the idea into production: patches of 14 pixels, merged 2×2, so one visual token covers a 28×28 pixel block, paired with M-RoPE which splits rotary position indices into temporal, height and width components so images and video share one positional scheme (Wang et al., 2024, arXiv:2409.12191). Pixtral 12B trains a vision encoder from scratch specifically to ingest images at their natural resolution and aspect ratio, and reports outperforming Llama-3.2 90B while being 7× smaller (Agrawal et al., 2024, arXiv:2410.07073).

Native resolution has one property the other two lack and it is a liability as much as a feature: token count is now a function of user input.

flowchart TB
    subgraph FIX[Fixed resize]
        F1[Squash to 336 square] --> F2[576 tokens always]
        F2 --> F3[Small text destroyed]
    end
    subgraph TILE[Tiling / AnyRes]
        T1[Pick grid by aspect ratio] --> T2[Encode each crop]
        T2 --> T3[Add global thumbnail]
        T3 --> T4[Seams cut through content]
    end
    subgraph NAT[Native resolution]
        N1[Keep aspect ratio] --> N2[Emit tokens per area]
        N2 --> N3[Cost set by uploader]
    end

    classDef amber fill:#b45309,stroke:#fbbf24,stroke-width:1px,color:#fff
    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef rose fill:#be123c,stroke:#fb7185,stroke-width:1px,color:#fff
    classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff
    class F1,T1,N1 amber
    class F2,T2,T3,N2 purple
    class F3,T4,N3 rose

[IMAGE: Three renderings of the same invoice line "Subtotal 1,428.60" as the model sees it after each policy: at 336px fixed resize the digits are a grey smear; at 2x2 tiling they are crisp but the string is split across a vertical seam; at native 1512px they are crisp and contiguous. Caption: "The three policies fail differently, and only one of the failures is visible in a benchmark score."]

Seeing It in Motion

A request through a VLM serving stack spends most of its life in the resolution policy's consequences, not in generation.

sequenceDiagram
    participant U as Client
    participant P as Preprocessor
    participant V as Vision encoder
    participant L as LLM prefill
    participant D as Decoder
    U->>P: 1512x982 PNG plus question
    P->>P: Select tile grid, normalise
    P->>V: 5 crops of 448 square
    V->>L: 1280 visual tokens
    U->>L: 24 text tokens
    Note over L: Prefill attends over 1304 tokens
    L->>D: KV cache built
    D->>U: 60 output tokens
    Note over V,L: Encoder cost is linear, prefill cost is quadratic

The note at the bottom is the part worth staring at. Teams optimise the vision encoder because it is the visible new component, but its cost is linear in tokens while the prefill it feeds is quadratic. At 1,300 tokens the encoder is still the larger term; by 10,000 tokens it is not close.

Token compression exists precisely to intervene between the two, and there are two moments to do it.

flowchart LR
    A[Encoder features] --> B{Compress where}
    B -->|Before LLM| C[Resampler or Q-Former]
    B -->|Before LLM| D[Pixel shuffle 4x]
    B -->|Inside LLM| E[Prune by attention]
    C --> F[Fixed small budget]
    D --> G[Fixed 4x reduction]
    E --> H[Input-dependent budget]

    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff
    classDef slate fill:#334155,stroke:#64748b,stroke-width:1px,color:#e2e8f0
    class A slate
    class B slate
    class C,D,E purple
    class F,G,H teal

The third branch is the interesting one. FastV observes that visual tokens receive very little attention mass in deeper layers of models like LLaVA-1.5 and QwenVL-Chat, learns attention patterns in early layers and prunes visual tokens in subsequent ones, letting a 13B model run at a 7B-like compute budget with better results, and reporting up to a 45× FLOP reduction for LLaVA-1.5-13B at the aggressive end of its configuration range (Chen et al., ECCV 2024, arXiv:2403.06764).

Read that finding backwards and it is an indictment of the resolution policy. If half the visual tokens can be dropped at layer 2 with modest damage, the policy generated tokens the model had no use for, and paid quadratic prefill cost to carry them.

Watch It Run

By the Numbers

One 1024×1024 screenshot, priced in visual tokens by seven systems.

Sources: OpenAI base and per-tile token rates and the high-detail resizing rule from the OpenAI images and vision guide; Gemini's 258-tokens-per-tile rule, the 384-pixel small-image threshold and the floor(min(w,h)/1.5) crop-unit formula from the Gemini image understanding docs; LLaVA-1.5 and LLaVA-NeXT from the papers and project blog cited above; InternVL 1.5 tile and pixel-shuffle figures from arXiv:2404.16821; Qwen2-VL's 28×28-pixel token from arXiv:2409.12191. The Qwen2-VL figure is approximate because the implementation rounds image dimensions to multiples of the merged patch size. Vendor tokenisers change; verify against a live token-count endpoint before budgeting.

Two derived quantities matter more than the table itself.

Documents. A 40-page PDF rendered at 1024×1024 per page is 41,280 tokens through Gemini's tiler and 51,200 through InternVL 1.5. That is a substantial fraction of a 128k context spent before the question is asked, and it is why document pipelines retrieve pages rather than sending whole PDFs.

Video. Sampling one frame per second at the same resolution puts a one-minute clip at roughly 62,000 tokens on the Gemini accounting. Ten minutes does not fit in most context windows at any price. Every practical video system therefore subsamples frames, drops resolution, or merges tokens temporally, and the quality consequences of those three choices are not equivalent.

[IMAGE: Log-scale line chart, visual tokens versus input resolution from 336 to 4096 pixels, one line per policy: flat at 576 for fixed resize, staircase for tiled policies as tile counts step up, smooth quadratic for native resolution, with a horizontal dashed line marking a 4,096-token budget cap and dots where each policy crosses it. Caption: "Where each policy hits a fixed token budget."]

A Concrete Example

An invoice screenshot, 1512×982, going through an InternVL-1.5-style tiling policy with 448×448 tiles. The question is "what is the subtotal", and the subtotal sits in the right-hand column at roughly \(x = 1180\).

Step 1: aspect ratio matching. The image ratio is \(1512 / 982 = 1.54\). The policy searches candidate grids with tile counts from 1 to 12 for the one whose aspect ratio is closest. A 3×2 grid has ratio \(3/2 = 1.5\), closest to 1.54; a 2×1 grid gives 2.0 and a 4×3 gives 1.33, both worse.

Step 2: resize to the grid. The image is resized to \(3 \times 448 = 1344\) wide by \(2 \times 448 = 896\) tall. Note that this is a downscale of 11% horizontally, so the policy has already lost some detail before tiling; text at 11 pixels of cap height is now at 9.8.

Step 3: tile and encode. Six tiles of 448×448, each 32×32 = 1,024 patches at patch size 14, each folded by pixel shuffle to 256 tokens. Plus one 448×448 thumbnail of the whole page at 256 tokens.

Step 4: where the subtotal landed. The original \(x = 1180\) maps to \(1180 \times (1344/1512) = 1049\) after resize. Tile column boundaries sit at \(x = 448\) and \(x = 896\), so \(x = 1049\) falls in column 3, comfortably inside tile 3 of the top row. Good.

Step 5: where the currency symbol landed. The value is right-aligned and its label "Subtotal" starts at \(x = 985\) in the original, which maps to \(985 \times 0.889 = 876\). That is 20 pixels left of the boundary at 896. The word "Subtotal" is about 60 pixels wide after resize, so it spans \(x \in [876, 936]\) and is cut by the tile boundary. Tile 2 sees "Subt", tile 3 sees "otal", and neither encoder pass sees both, because there is no attention across tiles inside the encoder.

Step 6: what saves it. The thumbnail. At 448×448 the whole page is downscaled 3.4×, so "Subtotal" is present but at roughly 3-pixel cap height, unreadable as glyphs yet sufficient as layout evidence: the language model has a fragment "Subt" at a known position, a fragment "otal" adjacent, and a blurred token-shaped blob in the thumbnail at the matching location. It usually reconstructs the word. It reconstructs the number less reliably, because a number has no lexical prior to fall back on: "1,428.60" split as "1,4" and "28.60" can plausibly be reassembled as 428.60, and there is no dictionary to veto it.

That is the whole failure mode in one line. Tile seams are recoverable for words and dangerous for digits, and no aggregate benchmark score reports the difference because the failure is concentrated in exactly the fields nobody puts in a VQA dataset.

Where It Breaks

Seams are systematic, not random

The tile grid is chosen by aspect ratio, so for a given document template the boundaries land in the same place on every page. A payments team processing 100,000 copies of one invoice layout does not get a 2% random error; it gets a specific field wrong on a large fraction of documents, deterministically. Diagnosing this from an aggregate accuracy metric is nearly impossible, and the fix, jittering the tiling offset or overlapping tiles by a margin, costs tokens that nobody budgeted.

Position embeddings do not extrapolate for free

A ViT pretrained at 224 learned absolute position embeddings for a 16×16 grid. Interpolating to a 64×64 grid is a distribution shift the encoder never trained through. It works after fine-tuning at the target resolution and degrades without it, which is why raising resolution on an existing model is a retraining project rather than a config change, and why models designed for variable input use interpolatable or factorised position schemes from the start.

The cost function belongs to the user

With native resolution, whoever uploads the image chooses your bill. A single 4K photo can consume more context than an entire conversation, and an adversarial user can construct requests that maximise token count at trivial effort. Production systems therefore cap tiles or expose a resolution parameter, which is honest but means the cap silently reintroduces downsampling for precisely the documents whose resolution mattered. Serving APIs have converged on exposing this as a dial, which moves the decision to the caller without making it easier.

Attention dilution

Adding visual tokens does not add proportional influence. In deep layers, attention mass on visual tokens is small relative to their count, which is what makes aggressive pruning viable. A practical consequence: past some resolution, extra tokens increase cost and latency while barely moving accuracy, and where that point sits depends on the task. Chart reading saturates late; scene captioning saturates almost immediately.

Benchmark scores hide all of it

Standard multimodal benchmarks are dominated by natural images at moderate resolution, where every policy performs similarly. The gap opens on dense documents, screenshots and charts, which are underrepresented. Worse, a model can score well without using the image at all: GeminiPro reaches 42.9% on MMMU with no visual input, and across six benchmarks image-free LLMs beat random by more than 24 points on average (Chen et al., 2024, arXiv:2403.20330). MMMU-Pro, which filters text-solvable questions and adds a vision-only setting where the question is embedded in the image, drops scores by 16.8 to 26.9 points (Yue et al., 2024, arXiv:2409.02813). If your evaluation cannot distinguish "read the image correctly" from "guessed from the question", it certainly cannot compare resolution policies.

Video multiplies every mistake

Every per-image decision is multiplied by frame count in video. A policy that overspends by 400 tokens per image overspends by 24,000 tokens per minute at 1 fps. Frame subsampling interacts badly with the content: sampling at 1 fps misses a two-frame gesture entirely, and the failure is silent.

[IMAGE: Heatmap of per-field extraction accuracy on a fixed invoice template, rows = policy (fixed resize, 2x2 tiles, 3x2 tiles, native), columns = field (vendor name, date, line item description, quantity, unit price, subtotal, total). Numeric right-aligned columns should show a visible accuracy dip for the tiled policies whose seam falls in that column. Caption: "Tile seams produce field-specific, template-deterministic errors that aggregate accuracy hides."]

Alternative Designs

The resampler row deserves a note, because it is the design that keeps getting rediscovered. Flamingo's Perceiver Resampler and BLIP-2's Q-Former both impose a fixed token budget regardless of input, which is exactly right for interleaved multi-image contexts and exactly wrong for a dense page, where the fixed budget is a bottleneck that no amount of input resolution can widen. The field moved to variable budgets for a reason, and it is now partially moving back through learned merging, which is the same idea with the budget chosen per input.

How It Is Used in Practice

Document retrieval has already skipped the parse. ColPali stores page images as multi-vector embeddings from a VLM and retrieves by late interaction, reaching 81.3 average nDCG@5 on ViDoRe against 67.0 for an Unstructured pipeline with captioning plus BGE-M3, and indexing at 0.39 seconds per page against 7.22 (Faysse et al., ICLR 2025, arXiv:2407.01449). Its 1,024 patch vectors per page at 128 dimensions cost about 257.5 KB per page in float16, which is the same resolution-policy trade appearing as an index-size line rather than a token line.

Agent screenshots are the worst case. A computer-use agent sends a full screenshot on every step. At 1,032 tokens per 1024×1024 frame, a 40-step task spends 41,000 tokens on images alone, most of which differ from the previous frame by a cursor position. Cropping to the region of interest, diffing against the previous frame, and reducing resolution for steps that only need layout are the standard mitigations, and all three are resolution-policy decisions made at the application layer because the model does not make them for you.

Capacity planning needs a per-image token estimate. The practical discipline is unglamorous: log the token count of every image request, chart the distribution rather than the mean, and set the tile cap from the tail. Image token counts are heavy-tailed because users upload screenshots of screenshots. A system provisioned on the mean will fall over on the 95^th percentile.

Insights Worth Remembering

1. Resolution is a fourth-power decision, not a linear one. Tokens scale with area and prefill attention scales with tokens squared, so doubling the linear resolution multiplies prefill attention work by roughly 16. Any conversation about "just increasing resolution" that does not mention this is missing the cost term entirely.

2. The cheapest legible policy beats the most detailed one. Once text is legible, extra resolution mostly buys tokens the model does not attend to. The engineering target is the legibility threshold for your document class, found empirically, not the maximum resolution the API accepts.

3. Tile seams fail on digits and recover on words. Language priors reassemble split words and cannot reassemble split numbers. Any pipeline extracting numeric fields from templated documents should measure per-field accuracy, not aggregate accuracy, and should suspect the tile grid first when one column underperforms.

4. Native resolution transfers the cost decision to the user. That is a product decision disguised as an architecture decision. Caps are unavoidable, so the question is only whether the cap is chosen deliberately or inherited from a default.

5. Token compression is a second resolution policy. Pixel shuffle, resamplers and attention pruning all decide how many visual tokens the language model sees, at different points in the stack. Teams tune the first policy and forget the second, then wonder why higher input resolution changed nothing.

6. The vision encoder is not where the cost is. Encoder cost is linear in tokens; the prefill it feeds is quadratic. Optimising the encoder is optimising the smaller term at any serious token count.

7. Video is the same arithmetic with a multiplier. Every per-image inefficiency becomes a per-second inefficiency. A resolution policy that is merely wasteful for images is disqualifying for video.

Open Questions

Is there a principled way to choose resolution per image? Current policies pick a tile grid from aspect ratio alone, ignoring content. A page of large text and a page of dense tables get the same treatment. A cheap content-adaptive predictor, running before the expensive encoder, is an obvious idea; whether it can be trained to be reliably better than a fixed cap is not established, and the failure mode of under-provisioning is severe.

How much of the resolution gain is legibility versus more parameters spent on the image? Higher resolution simultaneously increases legible detail and increases the compute the model applies to the image. Published comparisons vary both at once. Cleanly separating them, by holding token count fixed while varying source resolution, has been measured only in narrow settings.

Can seam artefacts be removed without paying for overlap? Overlapping tiles cost tokens proportional to the overlap; cross-tile attention inside the encoder costs quadratic attention across the full patch set. Whether a cheap intermediate exists, such as a thin band of shared boundary tokens, is open, and it is a question with immediate commercial value for document processing.

Do models actually use high-resolution detail, or do they mostly use it to rule things out? The pruning results are consistent with two very different stories: that most visual tokens are redundant, or that they matter briefly in early layers and are then summarised. Layer-wise probing to distinguish these has been done at small scale, and the answer determines whether compression should happen before the language model or inside it.

Is there a resolution-equivalent of a scaling law? Nothing published gives an accuracy-versus-visual-token curve of the kind that exists for parameters and training tokens. Practitioners currently choose resolution by trial. A reliable exponent, even a task-specific one, would turn a guess into a calculation.

Sources and Further Reading

1. Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. (2021). "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale." ICLR 2021. arXiv:2010.11929
2. Radford, A., Kim, J. W., Hallacy, C., et al. (2021). "Learning Transferable Visual Models From Natural Language Supervision." ICML 2021. arXiv:2103.00020
3. Alayrac, J.-B., Donahue, J., Luc, P., et al. (2022). "Flamingo: a Visual Language Model for Few-Shot Learning." NeurIPS 2022. arXiv:2204.14198
4. Li, J., Li, D., Savarese, S., & Hoi, S. (2023). "BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models." ICML 2023. arXiv:2301.12597
5. Zhai, X., Mustafa, B., Kolesnikov, A., & Beyer, L. (2023). "Sigmoid Loss for Language Image Pre-Training." ICCV 2023. arXiv:2303.15343
6. Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023). "Visual Instruction Tuning." NeurIPS 2023. arXiv:2304.08485
7. Liu, H., Li, C., Li, Y., & Lee, Y. J. (2023). "Improved Baselines with Visual Instruction Tuning." arXiv:2310.03744
8. Dehghani, M., Mustafa, B., Djolonga, J., et al. (2023). "Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution." NeurIPS 2023. arXiv:2307.06304
9. Liu, H., Li, C., Li, Y., et al. (2024). "LLaVA-NeXT: Improved reasoning, OCR, and world knowledge." Project blog, January 2024
10. Chen, Z., Wang, W., Tian, H., et al. (2024). "How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites." arXiv:2404.16821
11. Wang, P., Bai, S., Tan, S., et al. (2024). "Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution." arXiv:2409.12191
12. Agrawal, P., Antoniak, S., Bou Hanna, E., et al. (2024). "Pixtral 12B." arXiv:2410.07073
13. Chen, L., Zhao, H., Liu, T., et al. (2024). "An Image is Worth ½ Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models." ECCV 2024. arXiv:2403.06764
14. Faysse, M., Sibille, H., Wu, T., et al. (2025). "ColPali: Efficient Document Retrieval with Vision Language Models." ICLR 2025. arXiv:2407.01449
15. Chen, L., Li, J., Dong, X., et al. (2024). "Are We on the Right Way for Evaluating Large Vision-Language Models?" arXiv:2403.20330
16. Yue, X., Zheng, T., Ni, Y., et al. (2024). "MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark." arXiv:2409.02813
17. OpenAI. "Images and vision" API guide, base and per-tile token rates. developers.openai.com
18. Google. "Image understanding," Gemini API documentation, tile and token accounting. ai.google.dev

Free to read, no ads, no sign-up. If it was useful you can buy me a coffee.