Interactions Across 3D and 2D

We need to build some interactions across the 2D screen and the 3D objects placed in our real world. We want to use the trees and bioswales we instantiated onto the map as buttons. Here is how it works.

1. To create a new empty game object in the scene and attach the RayCastInteraction component to it. This script will take charge of the interactions between the panel floating on the trees in 3D space and the 2D info panels on canvas. Then we drag the objects we need to fill the public fields. Read through the commented codes below and you will have an idea of what these public fields do.

//We need the main camera in the scene, which will be the start point of a detection ray cast
    public Camera camera;

    //Canvas Group is a way to set 2D panels that don't involve with GameObject.SetActive
    //tree info canvas group holds the info fields for tree info
    public CanvasGroup treeInfoCanvasGroup;
    //we need to hide the recalibrate group so we need to refer to it
    public CanvasGroup recalibrateGroup;

    //this be filled with the TreeInfoManager attached to the tree we clicked
    public TreeInfoManager treeInfoManager;

    //the two Text fields display the tree info
    //You can find these two objects under the TreeInfoDetails panel under the Main Canvas
    public Text treeDescription;
    public Text treeCN;

Then here is the method to make our 3D objects interactive buttons, read through the commented codes and you will know how it works.

    void Update()
    {
        //a ray is casted every frame from the mouse (finger) position
        Ray ray = camera.ScreenPointToRay(Input.mousePosition);

        //button 0 is the left button on a mouse and also the one-finger tap on a screen
        //when the user taps on the screen the codes below will run
        if (Input.GetMouseButtonDown(0))
        {
            //if the ray hits something, run the codes below
            if (Physics.Raycast(ray, out hit))
            {
                Transform objectHit = hit.transform;

                //if the item hit by the ray is tagged "Trees", run the codes below
                //you need to set the tree prefab to the tag "Trees"
                if(objectHit.gameObject.tag == "Trees")
                {
                    //Fill the blank info fields
                    treeInfoManager = objectHit.gameObject.GetComponent<TreeInfoManager>();
                    treeCN.text = treeInfoManager.commonName;
                    treeDescription.text = $"\nStructure of this tree: {treeInfoManager.treeStructure}" +
                                           $"\nCondition of this tree: {treeInfoManager.treeCondition}";

                    //Make the info panel on screen visible and interactable
                    treeInfoCanvasGroup.alpha = 1;
                    treeInfoCanvasGroup.interactable = true;
                    treeInfoCanvasGroup.blocksRaycasts = true;

                    //Hide the recalibrate group so the canvas is not too crowded
                    recalibrateGroup.alpha = 0;
                    recalibrateGroup.interactable = false;
                    recalibrateGroup.blocksRaycasts = false;
                }
            }
        }
        
    }

2. To change the tag of our ray cast target object. The prefab is in the folder Assets/Prefab/.

3. To add a 3D UI object to the tree prefab as the display panel of the tree info. Attach a FaceCam Script so the text will always face the main camera. This is an indication that the trees are interactive.