How facial recognition works in a video management system: detection, alignment, embeddings and 1:N matching, why the threshold matters more than any accuracy percentage, and what on-premise deployment changes.
A video management system performs 1:N identification, searching a face against a gallery of many identities, which is substantially harder than the 1:1 verification your phone does when it unlocks. Vendor figures often quote the easier task.
There is no single accuracy number for a face recognition system, because it does not have one operating point. It has a threshold, and that threshold is a trade off you choose based on what happens after a match.
Anything that acts automatically, such as opening a door, demands a considerably tighter threshold than an alert a human operator reviews before acting.
Capture quality and camera placement determine results more than model choice does. A well placed camera at face height on a choke point will outperform a better model badly mounted, every time.
Genuine self learning in this context means per camera threshold calibration and quality gated template refinement. It does not mean silently retraining a biometric model on your population, which Visylix deliberately does not do.
Facial recognition is a biometric technique that identifies or verifies a person by measuring the geometry and texture of their face and comparing that measurement against stored records.
The distinction that matters most, and that general coverage of the topic usually blurs, is between three genuinely different operations. Face detection means finding that there is a face in the frame, and where it is. It involves no identity whatsoever, and counting people, blurring faces for privacy, or triggering a recording are all detection tasks.
Face verification, or 1:1 matching, means confirming that a presented face matches one specific claimed identity. This is what a phone does when it unlocks. The system already knows who you claim to be and only has to agree or disagree.
Face identification, or 1:N matching, means searching a face against a gallery of many enrolled identities to find who it is, or to establish confidently that it is nobody in the gallery. This is what a video management system does, and it is materially harder than 1:1, because every additional enrolled identity is another opportunity for a false match.
A VMS deployment is almost always 1:N. Keep that in mind whenever you read a vendor figure, because 1:1 numbers are far more flattering and are frequently quoted in contexts where 1:N is what is actually being sold.
The pipeline is consistent across serious implementations, including ours, and it runs in five stages.
Detection comes first. A detector model scans each frame and returns bounding boxes for faces, along with a handful of landmark points, typically the two eye centres, the nose tip and the two mouth corners.
Alignment comes second. The face is warped onto a canonical reference layout using those landmarks, so that eyes and mouth sit in the same place for every face the system will ever compare. Without alignment, a head tilt alone changes the measurement more than a change of person does. This step is unglamorous and it does more for real world reliability than most model upgrades.
Embedding comes third. The aligned face crop passes through a recognition model that outputs a fixed length vector of numbers. In our implementation that is a 512 dimension, unit length vector. This vector is the faceprint. It is not a photograph and the original image cannot be reconstructed from it, though it remains biometric personal data under most regulatory regimes and must be treated as such.
Comparison comes fourth. The vector is compared against every enrolled vector in the gallery using cosine similarity, which measures the angle between two vectors and returns a score where higher means more similar.
Decision comes fifth, and this is where everything interesting happens. The best scoring gallery entry is accepted as a match only if its score clears a configured threshold. If nothing clears the threshold, the correct answer is no match, and a system that cannot confidently return no match is not fit for 1:N use.
There is no such thing as the accuracy of a face recognition system as a single number, because the system does not have one operating point. It has a dial.
Turn the threshold down and the system matches more readily. You catch more of the people you are looking for, and you also produce more false matches where the system names the wrong person. Turn the threshold up and false matches become rare, but the system increasingly fails to recognise people it should. You cannot reduce both errors at once by adjusting the dial. You can only choose where on the trade off you want to sit, and that choice depends entirely on what happens next.
The two errors are not symmetric in consequence, and the right setting follows from asking who bears the cost of each. For a watchlist alert that a human operator reviews before any action is taken, a somewhat looser threshold can be reasonable, because the operator is the safeguard against a false match. For anything that acts automatically, such as opening a door, the threshold must be considerably tighter, because a false match is an unauthorised entry with nobody in the loop.
In Visylix the similarity threshold is a deployment setting rather than a fixed constant. The shipped default sits at a conservative starting point, and the documentation is explicit that it is a starting point only and should be field calibrated per deployment against real, consented pairs, with a distinctly tighter setting recommended for higher security access control use. We would rather say that plainly than imply that one number works everywhere.
This is also why we do not publish a headline accuracy figure for face recognition. A percentage is only meaningful alongside the threshold, the gallery size, the camera angles, the lighting and the demographics of the population it was measured on. Change any of those and the number changes. Published benchmark scores are useful for comparing models to each other under identical conditions, but they do not predict what will happen in your lobby.
There are five questions worth asking a vendor instead of asking how accurate it is. At what threshold was that figure measured, and what was the false match rate at that same threshold? Was it 1:1 or 1:N, and with a gallery of how many identities? What were the capture conditions, and how do they compare with a camera mounted above head height in mixed lighting? What was the demographic composition of the evaluation set? And can the threshold be calibrated per camera, or is it one global setting? A vendor who answers those five specifically is worth more of your attention than one who quotes a bigger number.
The failure modes are mostly about capture rather than about the model, and they are worth knowing in rough order of how much damage they do in real deployments.
Pose is the biggest. A face turned away from the camera loses the landmark geometry that alignment depends on. This is the single largest cause of missed recognition on CCTV, and it is a mounting problem rather than a software problem, because cameras placed high for wide coverage look down at the tops of heads.
Resolution at the face is next. What matters is not the megapixel count of the camera but how many pixels land across the face at the point of capture. A 4K camera covering a wide lobby can deliver fewer usable face pixels than a 2MP camera aimed at a doorway.
Motion blur and low light follow. Long exposure in dim conditions smears facial texture, and infrared illumination at night changes the appearance of skin in ways that reduce reliability.
Then come occlusion from masks, heavy glasses, helmets, hair and scarves, each of which removes part of the signal; ageing and appearance change between the enrolled photograph and the live face, which degrades gradually over months and years; and twins and close relatives, which remain genuinely hard and are not solved by any current system.
The practical consequence is that camera placement and enrollment quality determine your results more than model choice does. A well placed camera at face height on a choke point with a good enrollment gallery will outperform a better model badly mounted, every time.
If you accept every face the detector finds, you will enroll bad templates and generate matches on faces that carried too little information to justify one. Serious systems therefore score how recognisable a face is before they act on it, and there are two complementary ways to do this. Visylix uses both.
Geometric quality scores the obvious: detector confidence, the size of the face in pixels, sharpness, and head pose. It is cheap to compute and it catches the blatant cases.
Learned quality scores recognisability from the embedding itself, using a small model trained to predict how separable a given face genuine matches are from its impostor matches. This catches what geometry cannot, because a face can be sharp, frontal and well lit and still carry too little distinguishing information to be matched safely. In our implementation this runs on the embedding that recognition has already computed, so it costs microseconds and requires no extra inference pass.
Those scores are then used as gates, with deliberately different strictness. Enrollment has a quality floor, because a weak enrollment image becomes a permanent bad template that degrades every future match against that identity. Automatic template updates have a stricter floor than enrollment, because template drift compounds over time: one bad update is recoverable, but a slow slide is not.
A related detail is worth knowing. A presentation attack check gates enrollment. If someone can enroll a printed photograph or a face displayed on a screen, they have poisoned that gallery entry for every future comparison, so screening at the point of enrollment closes the most damaging path. Live CCTV matching remains annotate only rather than gating, since demanding a liveness check on every passing face in a corridor is not workable.
Live video offers something a still photograph does not, which is many looks at the same person.
Rather than picking the single best frame of a face as it crosses the scene and matching on that alone, a stronger approach is to fuse the embeddings from across the whole track, weighted by how recognisable each frame was. Frames where the subject was turned away or blurred contribute little, and the clear frontal moment contributes most.
The result is a more stable representation than any single frame gives, which matters a great deal on CCTV where any individual frame may be poor. Visylix does this by default, falling back to best frame matching when a track has not yet accumulated enough evidence to fuse.
Self learning is used loosely enough in this market that it is worth being precise about what does and does not happen.
Two things genuinely adapt in Visylix. The first is per camera threshold calibration. Cameras differ, and a camera facing a bright window at four in the afternoon produces a different distribution of similarity scores than one in a windowless corridor, so a single global threshold is wrong for most cameras in any estate. A nightly job fits the distribution of non match scores observed at each individual camera and sets that camera effective threshold at a configured target false positive identification rate, using the globally configured threshold as a floor so that calibration can tighten a camera but never loosen it below the operator setting. It requires a minimum number of observations before it will act on a camera at all.
The second is template refinement. When a high confidence, high quality match occurs, the stored template can be nudged toward the new observation using a slow exponential moving average, so that gradual appearance change does not degrade recognition over time. The learning rate is deliberately low and the quality gate is deliberately strict, for the drift reasons described above.
What does not happen, by design, is automatic retraining of the recognition model on your faces. Visylix includes an opt in site adaptation capability that clusters unlabelled face embeddings from a site own cameras into high confidence pseudo identities, and it deliberately stops there: it produces the clusters and a governance summary, and it never trains and never swaps model weights. The internal fine tuning queue explicitly refuses face recognition jobs on biometric privacy grounds.
That is a considered position rather than a missing feature. Adapting a face recogniser on a site own population requires retaining face imagery for training, which is a decision with legal and consent consequences that belongs to the customer and their counsel rather than to a background job. And the failure mode is severe and irreversible: if the clustering merges two different people into one pseudo identity, fine tuning pulls their embeddings together and permanently degrades 1:N matching at that site. The clustering is correspondingly conservative and will discard a genuine identity rather than admit an ambiguous one.
If a vendor tells you their face recognition learns your people automatically and gets more accurate every day, ask them what happens when it learns two people as one.
A match is only useful if the system knows what that identity means to you. Enrolled faces are organised into groups that carry a category, so that a recognised face can be treated as blocked, watched, allowed or VIP, and raise an appropriately elevated alert rather than a generic notification.
That distinction drives real operational differences. A VIP arrival at a hotel entrance and a barred individual at the same door are the same technical event and completely different responses.
In Visylix a face match is an event like any other, which means it flows into the wider automation layer rather than sitting in a silo. Sudarshan, the live intelligence layer, surfaces recognition on the focused feed with severity rings and an alert rail, and lets an operator describe in plain language what to watch for. Sudarshan Rules can combine a face match with other conditions, in ordered sequences and with per condition branching, including correlation against access control events, so that a recognised face at a door with no accompanying badge event becomes a rule rather than a manual observation. Rules notify and integrate; they do not actuate hardware directly. Radha, the on premise AI copilot, can be asked about recognition events conversationally and can take action through its tool set, with destructive operations confirmation gated and every action audit logged.
One thing Visylix deliberately does not offer is a face based attendance module. Face recognition for time and attendance is a common request in the Indian market, and it is a different product with different consent, payroll and labour law obligations. We do not ship one and we do not market the capability as though we do.
Facial recognition attracts more regulatory attention than any other video analytic, and correctly so. The technical design has direct compliance consequences.
On data residency, face embeddings are biometric personal data. Under India Digital Personal Data Protection Act, the treatment of biometric data as a special category under the EU General Data Protection Regulation, and comparable regimes elsewhere, sending them to a third party cloud is a processing decision that requires justification, a lawful basis, and usually a contract specifying exactly what the processor may do. This is where deployment architecture stops being an IT preference. Visylix runs entirely on the customer own infrastructure as a container, including recognition and the copilot, with no cloud dependency and support for air gapped operation, so faces enrolled at a site remain at that site. That does not make a deployment lawful by itself, but it removes the cross border transfer question, which is often the hardest part of an assessment to satisfy.
On bias and differential performance, independent evaluation, most systematically by the US National Institute of Standards and Technology in its ongoing face recognition evaluation programme, has repeatedly found that error rates vary by demographic group and that the variation differs substantially between algorithms. This is a real and measured phenomenon rather than a hypothetical one. Two implications follow: a single global accuracy figure hides it entirely, and any deployment making consequential decisions about people needs human review and an appeal route.
On proportionality, the question that regulators and works councils ask first is usually not whether it is accurate but why you need it in this particular place. Face recognition on a secure server room door is straightforward to justify. The same technology across a general office floor rarely is. Scope it to the places where it is defensible.
A practical governance checklist is short. Establish and document the lawful basis before enrollment rather than after. Set a retention period for embeddings and enforce it technically. Restrict who can enroll, view and export gallery entries, and audit those actions. Post notice where recognition is operating, as most jurisdictions require. Keep a human in the loop for consequential decisions, with a route to challenge a match. Calibrate and record thresholds per camera, and keep that record. And review the gallery periodically, removing identities that no longer have a basis to be there.
A camera captures a face, software locates it and straightens it using landmarks such as the eyes and nose, a model converts the straightened face into a list of numbers that describes it, and that list is compared with lists stored for known people. If the closest stored list is similar enough to pass a threshold you have set, the system reports a match.
It depends on the threshold you set, the size of your gallery and the conditions at the camera, so the question cannot honestly be answered with a single number. A system tuned to rarely miss anyone will produce more false matches, and one tuned to rarely misidentify will miss more people. Ask instead what the false match rate is at the setting you intend to run, and evaluate on your own cameras rather than trusting a benchmark from someone else data.
Detection finds that a face is present and where it is, with no identity involved at all. Recognition compares that face against enrolled records to determine who it is. Detection alone is sufficient for people counting, privacy blurring and occupancy analytics, and it carries far lighter regulatory obligations because it does not process biometric identifiers.
Often yes, but placement decides it. The requirement is enough pixels across the face at a reasonable angle. Cameras mounted high for wide area coverage frequently do not deliver that, because they look down at the tops of heads. A recognition deployment usually adds a small number of well placed cameras at entrances rather than relying on the whole existing estate.
Poorly, unless the scene is illuminated. Infrared illumination allows detection at night but changes facial appearance in ways that reduce recognition reliability compared with daylight, because most recognition models are trained largely on colour imagery. Entrances intended for recognition should have adequate visible lighting.
Without a presentation attack check, yes. This matters most at enrollment, because a spoofed enrollment image corrupts that gallery entry permanently and poisons every future comparison against it. Screening at the point of enrollment against printed and screen displayed faces closes the most damaging version of this attack.
No. It can run entirely on your own infrastructure. Visylix runs recognition on premise inside the customer own deployment, including in air gapped environments, so enrolled faces and their embeddings never leave the site.
Matching itself needs only the numeric embedding, and the original image cannot be reconstructed from it. However, embeddings are still biometric personal data under most regulatory regimes and attract the same obligations, so this reduces exposure rather than removing the compliance question.
No. Face based time and attendance is a distinct product with its own consent, payroll and labour law requirements, and Visylix does not ship an attendance module.