Unity3D

How to create an AR videoplayer using Unity + Vuforia

How to create an AR video player using Vuforia Cloud Recognition

How to create an AR videoplayer using Unity + Vuforia

Introduction:

In this tutorial, we will learn how to play a video in AR through scanning an image with a URL using Vuforia 7.2.24 Cloud Recognition and unity 2018.2.1f1. We will use the video component of unity with the source set to URL and this URL will be taken from the cloud database of Vuforia Developer Portal.

Cloud Recognition is different from normal recognition because in normal recognition the image is stored in device databases but in cloud recognition, the image is stored in cloud databases.

Creating File for MetaData:

In Vuforia, every Cloud Image Target has an associated Metadata. Target metadata is nothing else than a custom (user-defined) string, i.e. a blob of text that you can fill with any text you like. In our case, we will enter the URL of the video we want to play.

So, we need to create a text file that contains a URL. This file will then be used as MetaData to play the video through the cloud.

First, open notepad and copy the following URL in it.

http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/TearsOfSteel.mp4

How to create an AR videoplayer using Unity + Vuforia

now save this file as .txt format.

Adding Cloud Database:

We need to add a cloud database in the Vuforia Developer portal. This database will contain

  • An image which will be scanned by our app.
  • MetaData file which is the text file we created that contains the URL.

Open Vuforia developer portal and go to the License Manager Tab in the developer tab and get your license key from there.

Adding cloud database

Now go to the target manager tab in Vuforia developer portal and click add database and add a database.

 

Now, select your database and then add a target.

 

Upload any image you want for target image enter 10 in width and in metadata package upload the text file that we saved earlier then click add

 

Setting Up Unity for Vuforia:

We need to set up unity so that it can use Vuforia plugin. Make sure that you have installed the Vuforia package in Unity.

Open a new unity project and switch platform to android

 

Select player settings and then in XR settings select Vuforia Augmented Reality

Using Vuforia:

Adding AR Camera:

To use Vuforia we don’t need the default camera used in unity, instead we will use an AR Camera will be used.

Delete the main camera and Add AR Camera by right-clicking in the hierarchy then Vuforia -> AR Camera.

Adding License Key:

Search for Vuforia configuration in project panel select it and add your app license key.

 

Adding Cloud Recognition:

The cloud recognition object is responsible for playing the video and handling errors also.

Add cloud recognition object in your scene by right click in hierarchy Vuforia->Cloud Image->Cloud Provider.

Adding Cloud Database Keys:

Select cloud recognition object and add your client access and secret keys that you can get from the access keys tab in Vuforia developer portal database

 

Adding Cloud Image Target:

The image target is the object that gets scanned in order to play the video.

Add a cloud Image target in your scene by right click in the hierarchy ->Vuforia->Cloud Image->Cloud Image Target.

Adding a Video Player:

The video player will use the URL and display the video.

Add a quad in Image Target’s child by right-clicking on Image Target -> 3D Object-> Quad;

Select quad and change its transform according to the following image in this way the Target Image won’t be blocking our video player.

Add a video player component to the quad

Set video player source to URL and audio output mode to audio source

Add an audio source to the quad and drag and drop the quad into the audio source of the video player component.

Now, change the quad’s name and tag to “Player”.

Scripting:

Create a C# script and name it “SimpleCloudHandler”

Open the script and add the “Using Vuforia;” to add the Vuforia library.

we will need to inherit our script from the “IcloudRecoEventHandler “ interface so that we can handle Cloud Reco Events.

Now, implement all the methods of the interface

public void OnInitError (TargetFinder.InitState initError){}
public void OnInitialized (){}
public void OnNewSearchResult (TargetFinder.TargetSearchResult targetSearchResult){}
public void OnStateChanged (bool scanning){}
public void OnUpdateError (TargetFinder.UpdateState updateError){}

OnInitError(TargetFinder.InitState initError): It returns a specific error whenever we encounter during initialization.

OnInitialized (): called when the CloudRecoBehaviour has finished initializing.

OnNewSearchResult (TargetFinder.TargetSearchResult targetSearchResult): returns a new search result whenever the app finds a new target.

OnStateChanged (bool scanning): Called when the CloudRecoBehaviour starts or stops scanning.

OnUpdateError (TargetFinder.UpdateState updateError): returns an error whenever we encounter an error during the running state of the app.

Now make variables for ImageTargetBehaviour, CloudRecoBehaviour and MainPlayer GameObject.


public ImageTargetBehaviour behaviour;
CloudRecoBehaviour cloud;
public GameObject mainPlayer;
public Text ErrorTxt;
string errorTitle,errorMsg;

Now in Start().


void Start () {
CloudRecoBehaviour cloudReco = GetComponent<CloudRecoBehaviour> ();
if (cloudReco) {
cloudReco.RegisterEventHandler (this);
}
cloud = cloudReco;
mainPlayer = GameObject.Find ("Player");
Hide (mainPlayer);
}

Explanation:

Here we have the reference of CloudRecoBehaviour attached to our gameobject. We have stored it in cloudReco variable which is then passed to cloud variable.

Then, we  the player object. we will store it in the mainPlayer variable. we are going to create a hide method which will hide the mainPlayer so that it does not show in our mobile screen.

Now in OnStateChanged(bool scanning).


public void OnStateChanged (bool scanning){
if (scanning) {
ObjectTracker tracker = TrackerManager.Instance.GetTracker<ObjectTracker> ();
tracker.TargetFinder.ClearTrackables (false);
}
}

Here we are checking if the program is still scanning and if it is we will clear all the trackable.

Now add these lines in the OnNewSearchResult function and add the “Using UnityEngine.Video;” also

 


public void OnNewSearchResult (TargetFinder.TargetSearchResult targetSearchResult){
GameObject newImageTarget = Instantiate (behaviour.gameObject) as GameObject;
mainPlayer = newImageTarget.transform.GetChild (0).gameObject;
GameObject augmentation = null;
if (augmentation != null) {
augmentation.transform.SetParent (newImageTarget.transform);
}
if (behaviour) {
ObjectTracker tracker = TrackerManager.Instance.GetTracker<ObjectTracker> ();
ImageTargetBehaviour imageTargetBehaviour = (ImageTargetBehaviour)tracker.TargetFinder.EnableTracking (targetSearchResult, newImageTarget);
}
string URL = targetSearchResult.MetaData;
mainPlayer.GetComponent<VideoPlayer> ().url = URL.Trim();
cloud.CloudRecoEnabled = true;
}

Here we created a new gameobject for our image target and also enabled the objectTrakcers and then we took the URL from metadata file and gave it to the mainplayer‘s video player and then enabled the cloud recognition.

Now in the Hide function.


void Hide (GameObject ob) {
Renderer[] rends = ob.GetComponentsInChildren<Renderer> ();
Collider[] cols = ob.GetComponentsInChildren<Collider> ();
foreach (var item in rends) {
item.enabled = false;
}
foreach (var item in cols)
item.enabled = false;
}

Here we are disabling the colliders and renders of our mainplayer so that it does not appearonn the mobile screen.

Now add this script to CloudRecognation Object

Now drag and drop the image target and player in the script fields

Testing:

Now build and run the app on your mobile and scan the image then the selected video will start playing.

Conclusion:

In this tutorial, we learned how to play a video in AR through scanning an image with a URL using Vuforia Cloud Recognition and unity. We used the video component of unity with the source set to URL. This URL was taken from the cloud database of Vuforia Developer Portal. You can download the whole project from our GitHub repository.

Reference:

For more information on using Unity with Vuforia visit the following Link.

https://docs.unity3d.com/Manual/vuforia-sdk-overview.html

https://library.vuforia.com/content/vuforia-library/en/reference/unity/index.html

Click to comment

Most Popular

To Top