React liveness/provide default device info #6633

riasatali42 · 2025-07-22T12:59:39Z

Description of changes

This PR introduces features to pass in and out the device information (device Id, device label, kind, group Id) for the camera of the liveness check. Also, provides support to add callback methods for camera changes or default device not found.

Key additions:

Pass Default device id and label so that the device will be automatically selected.
If device Id and device label both are present, device label will be prioritized.
Pass out device info (device Id, device label, group Id, kind) once the liveness check is completed.
Callback method added for default device not found if the device with the passed in device id/ label not found.
Callback method added for camera changes if the camera is changed. So that we can track the updated camera device with device info.

Issue ##6587, Pass deviceId in and out of liveness check

Description of how you validated changes

Created unit tests to test the added features.

Checklist

Have read the Pull Request Guidelines
PR description included
yarn test passes and tests are updated/added
PR title and commit messages follow conventional commit syntax
If this change should result in a version bump, changeset added (This can be done after creating the PR.) This does not apply to changes made to docs, e2e, examples, or other private packages.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…ice automatically

…mera is found

…hing

changeset-bot · 2025-07-22T12:59:43Z

🦋 Changeset detected

Latest commit: 3aa0cc7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@aws-amplify/ui-react-liveness	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Simone319 · 2025-07-22T14:48:59Z

Hello @riasatali42 , thanks for creating the PR! There's conflicts in the branch. Could you resolve the conflict and resubmit the PR?

…aNotFoundCallback and device change callback methods after resolving merge conflicts

riasatali42 · 2025-07-28T13:08:37Z

Hello @Simone319 , I have updated the branch with the current main branch and resolved all the conflicts. I've updated this PR instead of creating a new branch. Please let me know if I should do otherwise.

…github.com/SmarterServices/amplify-ui into react-liveness/provide-default-device-info

riasatali42 · 2025-07-29T13:49:47Z

@Simone319, I saw after running the tests that there were some ESLint issues. I've fixed those and updated the branch.
cc. @jasonfill

Simone319 · 2025-07-30T13:32:30Z

...es/react-liveness/src/components/FaceLivenessDetector/LivenessCheck/LivenessCameraModule.tsx

+              data: { newDeviceId: selectedDeviceId, newStream },
+            });
+          } catch (error: any) {
+            throw new Error('Error updating device:');


Let's include the original error message. ex:

throw new Error(`Error updating device: ${error.message}`);

Simone319 · 2025-08-01T12:27:29Z

packages/react-liveness/src/components/FaceLivenessDetector/service/machine/machine.ts

+            onCameraChange(deviceInfo);
+          }
+        } catch (callbackError) {
+          // console.error('Error in onCameraChange callback:', callbackError);


Here, we call still log the error, so that the error is not silently swallowed. However, we need to disable the eslint rule, such as:

// eslint-disable-next-line no-console console.error('Error in onCameraChange callback:', callbackError);

Simone319 · 2025-08-01T12:30:08Z

packages/react-liveness/src/components/FaceLivenessDetector/service/machine/machine.ts

+          // console.error('Error in onCameraNotFound callback:', callbackError);
+          // Don't rethrow to prevent state machine from getting stuck


Same as above.

Simone319 · 2025-08-01T13:31:01Z

packages/react-liveness/src/components/FaceLivenessDetector/service/machine/machine.ts

+
+        try {
+          const deviceInfo = getSelectedDeviceInfo(context);
+          onUserCancel(deviceInfo);


What's the use case of deviceInfo in onUserCancel callback?

The main goal was to make all the callbacks more consistent. I've discarded it as we won't need this.

Simone319 · 2025-08-07T12:28:00Z

...es/react-liveness/src/components/FaceLivenessDetector/LivenessCheck/LivenessCameraModule.tsx

  });

+  // Update the device if the selectedDeviceId changes
+  React.useEffect(() => {


Why do we need to create a new effect here instead of reusing the existing onCameraChange callback?

I think we can discard that as the onCameraChange is already handling everything. So, I discarded.

Simone319 · 2025-08-07T12:40:43Z

packages/react-liveness/src/components/FaceLivenessDetector/service/machine/machine.ts

+          const deviceInfo = getSelectedDeviceInfo(context);
+          onUserCancel(deviceInfo);
+        } catch (callbackError) {
+          // console.error('Error in onUserCancel callback:', callbackError);


Same as above

…rom the onUserCancel callback

riasatali42 · 2025-08-07T16:35:14Z

Hello @Simone319 , I've updated the PR as per the feedback.
cc. @jasonfill

adrianjoshua-strutt · 2025-08-12T15:29:07Z

Hi @riasatali42,

thank you for addressing our concerns. Could you add end-to-end tests to verify the behavior of your changes?

This guide should help you getting started: https://github.com/aws-amplify/amplify-ui/tree/main/packages/e2e

If you got any questions, feel free to reach out.

Thank you!

riasatali42 · 2025-08-15T18:05:40Z

Hello @adrianjoshua-strutt ,
I followed all the steps to configure and successfully pushed to amplify using "amplify push". But when I'm running the tests, I'm seeing this issue on this API endpoint:
api: POST /dev/livenessnolight/create?challengeType=FaceMovementAndLightChallenge
RESPONSE: 502 {"message": "Internal server error"}

Could you please guide me here to resolve this issue?

soberm · 2025-08-22T15:39:31Z

Hi @riasatali42,
sry for the delay. The E2E environment for liveness requires some changes such that it works for external contributors. Its missing the permissions for the Lambdas to call Rekognition service (thats the reason for the 502 you are seeing) and needs to be switched to use the prod endpoint (see #6673). Once this is merged you should be able to run the tests.

riasatali42 · 2025-08-27T12:55:28Z

Thanks @soberm for your reply. Should I merge this branch (#6673) with my branch to test the e2e testing? I've added the rekognition permissions with the IAM user (the one I'm using for the lambda).

soberm · 2025-08-29T11:23:58Z

Hi @riasatali42,
I already merged PR #6673 to main. If you update your branch with the main branch you should now be able to run the tests with your personal backend once you deploy the backend changes. Please let us know if you need any assistance.

…ide-default-device-info

riasatali42 · 2025-09-01T12:49:06Z

Hello @soberm ,
Thank you so much, all of those issues are resolved now. I can run the test cases properly. But for some reason, once I run the e2e test case on liveness such as 'disable-start-screen', It's saying 'Camera is not accessible'. So, the test case is not passed. If I open the link on a browser, I can see the camera working properly.
Could you please help me here or give me any hint on why this could show this screen?

soberm · 2025-09-02T09:06:36Z

Hi @riasatali42,
I checked out your PR but unfortunately I don't see this issue on my side. Maybe there is a problem with your browser permissions.

…ss default device pass in

riasatali42 · 2025-09-03T12:24:11Z

Hello @soberm , I've updated the PR. Please let me know if we need anything else.

soberm · 2025-09-03T12:43:39Z

Hello @soberm , I've updated the PR. Please let me know if we need anything else.

Hey @riasatali42, I can't execute the E2E tests due to a missing step implementation.

Step implementation missing for "I see the "Device ID" selectfield".

…ide-default-device-info

… case for device info pass out

riasatali42 · 2025-09-04T13:45:14Z

Hello @soberm , I've added the step. Please take a look now.

jasonfill · 2025-09-10T14:11:40Z

Hello @soberm - circling back on this. We are eager to get this merged. Could the team please review and return any issues to us so we can bring this to a close?

…ide-default-device-info

soberm · 2025-09-16T13:05:06Z

Hello @soberm - circling back on this. We are eager to get this merged. Could the team please review and return any issues to us so we can bring this to a close?

We're currently reviewing this internally with our team and update you here once we have more information.

soberm · 2025-09-23T09:30:35Z

Hi @riasatali42, @jasonfill,

thank you for your work on implementing this feature for the FaceLivenessDetector component. We appreciate the implementation and thorough testing you've provided. Following our internal API review, here are the key changes needed to align your implementation with our design recommendations. Please let us know if you have any concerns or questions about these requested changes - we're happy to discuss the rationale behind any of these recommendations.

Requested Changes

Simplify Device Input Parameters

Your PR includes both deviceId and deviceLabel with prioritization logic. The device selection should be handled by looking up deviceId by label externally, keeping the API surface simple.

We suggest removing the deviceLabel parameter and only support deviceId in the FaceLivenessDetectorCoreConfig:

export interface FaceLivenessDetectorCoreConfig {
  deviceId?: string; // Only this parameter should be supported
  // ... other existing config options
}

Remove Additional Callback Methods

We suggest removing the callbacks that were added:

onDefaultDeviceNotFound
onCameraChange

The existing onError and onAnalysisComplete callbacks already provide the necessary mechanism to return device information when the liveness check succeeds or fails. Adding new callbacks complicates the API surface.

Update Existing Callback Signatures

We suggest using the existing callbacks to include optional DeviceInfo parameter to pass this information:

export interface DeviceInfo {
  deviceId: string;
  groupId: string;
  label: string;
}

export interface FaceLivenessDetectorCoreProps {
  onAnalysisComplete: (deviceInfo?: DeviceInfo) => Promise<void>;
  onError?: (livenessError: LivenessError, deviceInfo?: DeviceInfo) => void;
}

Ensure DeviceInfo Consistency

Ensure DeviceInfo properties match browser MediaDevices API standards (always strings, never null).

Thank you again for taking the initiative to implement this feature and for your patience as we work through the design.

jasonfill · 2025-09-24T12:58:06Z

@soberm I have a few items on these

Simplify Device Input Parameters
The issue here is that deviceId can change. So if we are storing the last device used (which is what we are going to do) so we can pre-populate it, the deviceId will not be reliable. We provided both options and prioritized the label to offer choices. If the team wants to drop one, I would suggest we drop the deviceId since it is more ephemeral than the label. Please guide the team on what they would like in this regard.
Remove Additional Callback Methods
We can see getting rid of onCameraChange, although there could be some use cases for this. However, if the team feels that it is too much extra, it can be removed. On the other hand, it makes a lot of sense and would be helpful to know if you're passing in a default device that device is not actually in the list of devices. onDefaultDeviceNotFound can be used for external cleanup or surfacing other messages.

Numbers 3 and 4 make sense, and we can incorporate them. Please provide the final direction on 1 & 2, and we will look to wrap this up.

cc @riasatali42

soberm · 2025-09-24T15:23:00Z

@jasonfill thank you for the feedback. Here are our thoughts on this:

1. Device Input Parameters
We understand your concern about deviceId being ephemeral. We want to avoid any ambiguity from using deviceLabel on the librarie's side since it's not unique compared to the deviceId according to the documentation. Then, consumers would have the responsibility to resolve this by mapping deviceLabel to the correct deviceId.

2. Callback Methods
Regarding onDefaultDeviceNotFound, we think this functionality can be better handled through the existing onError callback. Device not found scenarios are essentially error conditions, and we already return error types like CAMERA_ACCESS_ERROR through onError. This approach maintains consistency with our existing error handling patterns.

What are your thoughts on this approach?

jasonfill · 2025-09-25T11:51:23Z

@soberm since the deviceId is not reliable, it makes no sense for us to use that. Our goal is to preload the camera that was used last. The only level of reliability is to do that via label. If label is. Or permitted, this feature is rendered useless for us. Unless I am missing the concept of what you are outlining.

On point 2, that makes total sense. We can adjust to use the onError.

soberm · 2025-09-25T12:05:08Z

@soberm since the deviceId is not reliable, it makes no sense for us to use that. Our goal is to preload the camera that was used last. The only level of reliability is to do that via label. If label is. Or permitted, this feature is rendered useless for us. Unless I am missing the concept of what you are outlining.

On point 2, that makes total sense. We can adjust to use the onError.

@jasonfill Yes, the deviceId might change but if you already store the deviceLabel of the last used device it should be possible to get the DeviceInfo for the device with this label and then pass the deviceId to the liveness component. Maybe like this:

// Get device by label and extract deviceId
async function getDeviceIdByLabel(targetLabel) {
  const devices = await navigator.mediaDevices.enumerateDevices();
  const camera = devices.find(device => 
    device.kind === 'videoinput' && device.label === targetLabel
  );
  return camera?.deviceId;
}

const lastUsedLabel = getStoredCameraLabel(); // your stored device
const deviceId = await getDeviceIdByLabel(lastUsedLabel);

if (deviceId) {
  // Pass to liveness component
  <LivenessComponent deviceId={deviceId} />
}

Would this approach work for your use case ?

jasonfill · 2025-09-25T14:50:21Z

@soberm We can make that work. The team will roll these changes in ASAP.

Thanks for your time and collaboration.

jasonfill and others added 18 commits June 14, 2025 09:37

feat: add ability to pass device in and out of liveness check

58d9ba1

chore: clean up some formatting issues

f00e7fd

chore: fix additional formatting issues

16f1c07

chore: fix more formatting

1216c8f

chore: fix some formatting items

54d64f3

chore: additional formatting changes

afd93c1

chore: some resolutions were removed, replaced them

e8a5653

chore: formatting

b11b3ce

chore: remove duplicate items in package

85f47e0

chore: formatting

8ca425d

chore: add ability to provide default device label to select that dev…

933013b

…ice automatically

chore: added callback method for camera changed and if not default ca…

afb5077

…mera is found

chore: added selectableDevices array check

8c9f3e9

chore: added default device info in machine tests for unit testing

a89ff4d

chore: default device info for unit testing default device pass in

8905251

chore: unit tests added for default device not found and camera switc…

36e0fd3

…hing

chore: added device selection priority unit tests

497bfb0

docs: added changeset for minor changes

f0f7923

riasatali42 requested a review from a team as a code owner July 22, 2025 12:59

pranavosu added the external-contributor label Jul 22, 2025

Simone319 added run-tests Adding this label will trigger tests to run and removed external-contributor labels Jul 22, 2025

riasatali42 and others added 4 commits July 28, 2025 15:35

fix: resolved merge conflicts

3c75e1d

fix: fixed mock device info after resolving merge conflicts

0a3ad1b

chore: added updated unit tests for passed in device label, callCamer…

724dca3

…aNotFoundCallback and device change callback methods after resolving merge conflicts

Merge branch 'main' into react-liveness/provide-default-device-info

7a701c1

jasonfill mentioned this pull request Jul 28, 2025

Pass deviceId in and out of liveness check #6587

Open

2 tasks

riasatali42 added 3 commits July 29, 2025 19:18

fix: fixed ESlint issue on livenesscameramodule

d1bede1

fix: fixed eslint issue on adding type on error and replaced operator

af9b3bf

Merge branch 'react-liveness/provide-default-device-info' of https://…

bfd66d5

…github.com/SmarterServices/amplify-ui into react-liveness/provide-default-device-info

Simone319 reviewed Aug 7, 2025

View reviewed changes

fix: resolved PR feedback, added console error, removed device info f…

3662944

…rom the onUserCancel callback

Merge remote-tracking branch 'upstream/main' into react-liveness/prov…

6c3432d

…ide-default-device-info

chore: merged with main branch and added e2e testing for react-livene…

3cccaa0

…ss default device pass in

riasatali42 added 2 commits September 4, 2025 19:36

Merge remote-tracking branch 'upstream/main' into react-liveness/prov…

2792003

…ide-default-device-info

chore: Added step for I see the Device ID selectfield, added e2e test…

3634688

… case for device info pass out

Merge remote-tracking branch 'upstream/main' into react-liveness/prov…

3aa0cc7

…ide-default-device-info

		// console.error('Error in onCameraNotFound callback:', callbackError);
		// Don't rethrow to prevent state machine from getting stuck

React liveness/provide default device info #6633

Are you sure you want to change the base?

React liveness/provide default device info #6633

Conversation

riasatali42 commented Jul 22, 2025

Description of changes

Key additions:

Issue ##6587, Pass deviceId in and out of liveness check

Description of how you validated changes

Checklist

Uh oh!

changeset-bot bot commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Simone319 commented Jul 22, 2025

Uh oh!

riasatali42 commented Jul 28, 2025

Uh oh!

riasatali42 commented Jul 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

riasatali42 commented Aug 7, 2025

Uh oh!

adrianjoshua-strutt commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

riasatali42 commented Aug 15, 2025

Uh oh!

soberm commented Aug 22, 2025

Uh oh!

riasatali42 commented Aug 27, 2025

Uh oh!

soberm commented Aug 29, 2025

Uh oh!

riasatali42 commented Sep 1, 2025

Uh oh!

soberm commented Sep 2, 2025

Uh oh!

riasatali42 commented Sep 3, 2025

Uh oh!

soberm commented Sep 3, 2025

Uh oh!

riasatali42 commented Sep 4, 2025

Uh oh!

jasonfill commented Sep 10, 2025

Uh oh!

soberm commented Sep 16, 2025

Uh oh!

soberm commented Sep 23, 2025

Requested Changes

Uh oh!

jasonfill commented Sep 24, 2025

Uh oh!

soberm commented Sep 24, 2025

Uh oh!

jasonfill commented Sep 25, 2025

changeset-bot bot commented Jul 22, 2025 •

edited

Loading

adrianjoshua-strutt commented Aug 12, 2025 •

edited

Loading