Description
Summary # Project Scope: Camera-Based Self-Checkout Behavior Verification ## Context We want to explore a camera-based solution for self-checkout stations that can identify suspicious scan and payment behavior with high precision. The system should not try to detect every possible irregularity. The priority is to flag a limited number of clear, explainable cases that are useful for human review and that avoid unnecessary false positives. The first version should be suitable for a controlled pilot in a real store environment, using video from self-checkout stations and, where visible, the scanner’s good-read signal. The system should be positioned as a behavior verification and case-review tool, not as an automatic theft accusation system. ## Core Scenarios The solution should focus on two main types of behavior. ### 1. Product not properly scanned This includes two different situations: * a product is moved from basket, trolley, or holding area to the bagging area without passing through the scanner zone; * a scan-like gesture is made, but no matching scanner good-read signal is observed. The system should not rely on counting products from video. Counting is too fragile in a real store context because of rescans, voids, rearranging products, multiple items being held together, staff assistance, and occlusion. The preferred logic is based on visible actions, object movement, scanner-zone interaction, and matching with the scanner signal. ### 2. No observable payment gesture The system should detect whether the customer appears to present a card, smartphone, or similar payment object to the payment terminal during the session. This only validates the presence of a payment gesture. It does not prove that the payment was successful. Failed, cancelled, or rejected transactions require POS or payment-terminal integration and are not part of the first scope unless separately proposed. ## Expected Functional Scope The supplier should be able to propose an approach covering: * detection of a self-checkout session from arrival to departure; * configuration of zones per checkout station, including scanner, payment terminal, basket/trolley area and bagging area; * detection of hand/object movement around the scanner zone; * detection of object movement from basket/trolley area to bagging area; * detection of the scanner’s visible good-read signal; * matching scan gestures with good-read events within a configurable time window; * detection of payment gestures at the payment terminal; * generation of structured events and reviewable cases; * a minimal review or feedback mechanism for pilot validation. The first version should assume one camera stream per checkout station, with the full checkout station visible in the image. ## Technical Direction The preferred architecture is edge-based or on-premise processing, mainly for privacy, latency and bandwidth reasons. The supplier should describe the proposed setup, including video ingestion, processing pipeline, hardware needs and possible connection to existing camera or NVR systems. The expected pipeline includes: 1. video ingestion; 2. zone calibration per station; 3. occupancy and session detection; 4. hand/object movement detection; 5. scanner-zone interaction detection; 6. scanner good-read detection; 7. temporal matching between gesture and scanner signal; 8. movement analysis between configured zones; 9. payment gesture detection; 10. rule-based decision logic; 11. case creation and feedback capture. The system may use existing pretrained computer vision models for person detection, tracking, hand/keypoint detection, object-in-hand detection and movement analysis. A custom model trained from scratch is not expected in the first version unless the supplier considers it necessary after validation. The final decision logic should remain explainable. Each alert should be traceable to concrete observed events, such as: * “object moved to bagging area without crossing scanner zone”; * “scan gesture detected without matching good-read signal”; * “no payment gesture observed during the session.” ## Pilot Phasing The supplier should propose a phased approach. ### Phase 0: Validation Before building the full solution, the key assumptions should be tested on representative footage from a real or pilot store. This phase should validate: * camera angle and visibility; * scanner good-read visibility; * reliability of scan-gesture detection; * stability of the matching window between gesture and good-read; * product movement detection from basket/trolley to bagging area; * behavior under rapid scanning, lighting variation, rescans, voids and normal customer behavior. The outcome should be a clear go/no-go or scope adjustment. ### Phase 1: Detection Pipeline Build the core perception layer and structured event generation: * sessions; * zones; * scanner gestures; * good-read events; * product movement; * payment gestures. ### Phase 2: Decision Logic Add the rules and thresholds that convert observed events into suspicious cases. This includes confidence scoring, sensitivity settings and initial handling of known edge cases. ### Phase 3: Review and Feedback Provide a minimal way to review flagged sessions, inspect evidence and label cases as confirmed, rejected or uncertain. This feedback should support tuning during the pilot. ### Phase 4: Pilot Deployment Deploy on a limited number of checkout stations, calibrate per station, monitor results, tune thresholds and report performance. ## Deliverables Expected from Supplier The supplier should describe and price the following deliverables: * technical architecture; * required camera setup; * hardware recommendation; * video ingestion approach; * zone calibration method; * detection pipeline; * scanner good-read detection; * gesture-to-good-read matching logic; * movement analysis; * payment gesture detection; * rule-based decision layer; * case generation; * minimal review or feedback flow; * pilot deployment support; * validation and performance reporting; * future integration options. ## Key Assumptions The initial scope assumes: * one camera stream per checkout station; * camera view includes scanner, basket/trolley area, bagging area and payment terminal; * scanner good-read signal is visible in the video; * one customer or one shopping party normally uses the station at a time; * representative footage is available before implementation; * POS integration is not included in the first version; * payment success is not validated in the first version; * all alerts are reviewed by a human before action is taken. ## Important Edge Cases The supplier should explicitly explain how the solution handles or excludes: * staff assistance; * multiple people at one checkout station; * rescanning; * voiding; * items being returned; * products held together; * occlusion by hands, arms, bags or body position; * lighting changes; * scanner or terminal reflections; * checkout stations with different hardware behavior; * failed payment after a visible payment gesture. For each of these, the supplier should indicate whether it is handled in the first version, partially handled, excluded, or only possible with POS or payment integration. ## Success Criteria The pilot should mainly be judged on precision and operational usefulness, not on maximum detection volume. Relevant metrics include: * false positives per checkout station per day; * percentage of flagged cases accepted by reviewers; * quality and clarity of the explanation per case; * latency between session end and case availability; * stability across different checkout stations; * calibration effort per station; * hardware performance; * usefulness for store staff or loss-prevention teams. A smaller number of reliable cases is preferred over many uncertain alerts. ## Future Extensions Possible later extensions include POS integration, transaction validation, payment-outcome validation, staff recognition, better multi-person handling, mobile alerting, full case-management integration, reporting dashboards and model improvement based on reviewed pilot data. The first supplier proposal should clearly separate what is included in the pilot, what is optional, and what depends on later integrations.