Goals

• learn the basic concepts and important methods of MT4j's components
• learn how components in MT4j are created and destroyed
• learn how components can be made interactable
• learn how components can be displayed

Introduction

Components are the cornerstone of every application with a graphical user interface.

In MT4j, the graphical user interface is based on a hierarchic structure which allows the composition of user interface components in a tree structure often referred to as a 'scene graph'. Creating such a component tree is done by attaching components to other components, making the attached component the "child" of the other component which then becomes the "parent". A component can have one parent component at most while the number of child components is not limited.

Besides easier management of components and other advantages, this tree structure resembles components in the real world, where components are often made up of smaller components.

In MT4j, the base class for all components is the class MTComponent. This component class provides some basic functionality and has no direct visible representation. Thus this base class can be used as a group component containing other groups or visible components.

Each component, (amongst other things) stores its own, relative position/rotation/scale, parent component, list of children, input/gesture processors and listeners.

Creating components and creating a hierarchy

This example shows how to create a MTComponent instance and add it as a child to another component. The "app" parameter, that every component takes as an argument, holds the reference to our MTApplication instance (which is a instance of PApplet). It is internally used for positioning, drawing and other things.

MTComponent a = new MTComponent(app);
MTComponent b = new MTComponent(app);
a.addChild(b);
a.removeChild(b);

Adding a component as a child of another has several effects on the child component:

• Some settings of a component also affect all children of the component
  => e.g. setVisible() sets the visibility for the whole sub-hierarchy
• Transformations of parents effect the total transformation of the child components
• Children are updated and drawn after their parents. This will make children appear in front of their parents if they are drawn at the same distance from the camera (or if the depth buffer is disabled in OpenGL)
• Children are favoured at picking (hit-testing, selection) over their parents if they have the same distance from the camera

Relevant methods:

• addChild(MTComponent child)
  Adds the specified component as a child to our component
• removeChild(MTComponent child)
  Removes the specified component from the list of children of our component.
• addChildren(MTComponent[ children)
  Adds an array of components to our component
• getChildren()
  Returns an array containing all the component's children
• getParent()
  Returns the parent component of the component. May be null if the component has no parent.

Destroying components

If we definitely don't need a component anymore and we want to remove it from our application, the component's destroy() method should be invoked.

This has following effects on the component:

• the component is removed from its parent and by this also from the scene graph/canvas
• the component's destroyComponent() method is invoked
• destroy() is called on all children of the component so that recursively all children of the component are destroyed as well

The main reason why we should call the destroy() method instead of just removing the component from its parent is that some components allocate memory on the graphics card (textures for example) or in native code (especially when using the OpenGL renderer). This memory has to be freed manually since it cannot be reached by java's garbage collector. Freeing this memory is then supposed to be done in the destroyComponent() method. So if we create our own components and create OpenGL or other native objects, we should override this method and handle the memory freeing there. The built in components of MT4j already implement the destroyComponent() method where it is needed. Even if a component doesent use any resources that need manual cleanup it's best to develop the habit of using the destroy() method to remove components from the application.

Relevant methods:

• destroy()
  This will destroy the component and its children and remove every component from its parent. It also invokes destroyComponent() on every component.
• destroyComponent()
  This method is supposed to contain the code to clean up all (native) resources used by the component being destroyed. It is not supposed to be called by the user. Use destroy() instead.

Making components interactable

If we want to interact with components, there has to be a way of checking which component there is at a certain position on the screen. Since MT4j is also suitable for 3D applications this check is a bit more complex than in a strict 2D environment. MT4j does this by constructing a (pick-) ray starting from the camera going through the position on the screen we want to check at. Then we ask every component in our scene if it has an intersection with this ray. If a component intersects the ray the point of intersection is saved and compared to intersection points of other components. The component whos intersection point is nearest to the camera then becomes the target of the input. So in order to make components interactable we have to implement this ray intersection test. All shapes in MT4j have this intersection test implemented already. If we want to create custom components, we have to implement it ourselves by overriding the component's getIntersectionLocal(Ray ray) method which has to return the intersection point or null if there was no intersection.

Relevant methods:

• pick(float x, float y)
  This will check which component is at the specified screen position. The check starts at the component and also checks all its children.
• getInterSectionGlobal(Ray ray)
  This method takes a ray specified in global space coordinates and returns a hit point or null if no intersection occurred. The check starts at the component and also checks all its children. The method recursivly calls getInterSectionLocal() on all children.
• getInterSectionLocal(Ray ray)
  This checks only the component on which the method is invoked, not its children. The ray has to be in the components local space (See also globalToLocal(Ray ray)). [If we want to create a custom component with a custom ray intersection check we have to override this method and do our intersection test with the local ray in this method!]
• getComponentAt(float x, float y)
  This method is only available in the MTCanvas component. It works similar to the pick(x,y) method with the difference that will return the MTCanvas object if no component was hit by the ray. It also uses a cache to speed up intersection tests in a short time span.
• setPickable(boolean pickable)
  If we set pickable to false on a component, the component will never be checked for an pick-ray intersection and thus wont be recieving input events. For components that are only used for visualisation and not for interaction this should be always be set to "false". This will also increase the performance of the overall picking check process. (Default value is "true")
• ToolsInterSection.getRayPlaneInterSection(Ray ray, Vector3D planeNormal, Vector3D pointInPlane)
  This helper method calculates the intersection between a ray and a plane and can be useful, especially for 2D shapes that we know lie on a certain plane in our virtual world.

Recieving and handling input

If a component is interactable and has become the target of an interaction it will be sent input events. We can listen and react to these events by adding an input listener to the component and implement its processInputEvent method.

Example:

component.addInputListener(new IMTInputEventListener() {
        public boolean processInputEvent(MTInputEvent inEvt) {
                if (inEvt instanceof AbstractCursorInputEvt) { //Most input events in MT4j are an instance of AbstractCursorInputEvt (mouse, multi-touch..)
                        AbstractCursorInputEvt cursorInputEvt = (AbstractCursorInputEvt) inEvt;
                        InputCursor cursor = cursorInputEvt.getCursor();
                        IMTComponent3D target = cursorInputEvt.getTargetComponent();
                        switch (cursorInputEvt.getId()) {
                        case AbstractCursorInputEvt.INPUT_DETECTED:
                                System.out.println("Input detected on: " + target + " at " + cursor.getCurrentEvtPosX() + "," + cursor.getCurrentEvtPosY());
                                break;
                        case AbstractCursorInputEvt.INPUT_UPDATED:
                                System.out.println("Input updated on: " + target + " at " + cursor.getCurrentEvtPosX() + "," + cursor.getCurrentEvtPosY());                   
                                break;
                        case AbstractCursorInputEvt.INPUT_ENDED:
                                System.out.println("Input ended on: " + target + " at " + cursor.getCurrentEvtPosX() + "," + cursor.getCurrentEvtPosY());
                                break;
                        default:
                                break;
                        }
                }else{
                        //handle other input events
                }
        return false;
        }
});

This example shows how we would listen to different input events targeted at that component. The information carried by the event depends on the input event's type.

The component input processors used for multi-touch gesture recognition are in essence plain input listeners, too. How to recieve and react to gesture events is described in this tutorial.

Relevant methods:

• addInputListener(IMTInputEventListener i)
  Adds an input listener to the component.
• removeInputListener(IMTInputEventListener i)
  Removes an input listener from the component.
• getInputListeners()
  Returns an array of the attached input listeners of the component.
• setEnabled(boolean enabled)
  If the component is not enabled, it won't recieve input events.

Drawing and displaying components

Of course, one of the main uses of components is to display something on the screen. Components are drawn by calling their drawComponent method. Once we add a component as a child to our scene's canvas (or to any other component which already is added to the canvas) its draw method will be invoked automatically each frame.

To make a component draw something on the screen we simply override its drawComponent method and fill it with drawing commands. In the basic MTComponent class this method is empty. Since MT4j uses Processing undearneath we can use processing's syntax and drawing commands here. For that purpose, the draw method provides us with a PGraphics instance as an argument which is processing's graphics context.

Example:

public void drawComponent(PGraphics g){
        super.drawComponent(g);
 
        g.fill(255,0,0);
        g.stroke(0,0,255);
        g.triangle(30, 75, 58, 20, 86, 75);
}

This component would now draw a triangle with red filling and a blue outline at the specified coordinates. For a full list of processing's drawing commands take a look at the processing reference page.

When we use draw commands that require coordinates or dimensions as parameters as in the above example, we must be aware that we are specifying them in the components local, un-transformed coordinate space. This means that if we draw a rectangle with a width of 100 it may appear bigger or smaller on the screen, depending on the component's global transformation matrix. If we scale our component by 2 for example, the width of the rectangle will actually appear to be 200 on the screen. This way we dont have to draw the rectangle with different coordinates or dimensions each time we want it to appear differently but instead we just change the component's transform by translating, rotating or scaling the component. (See tutorial Component Transformations)

Relevant methods:

• drawComponent(PGraphics g)
  Draws the component. The children of the component are not drawn by this method.
• setVisible(boolean visible)
  Sets the visibility of the component and all its children.

Updating components

Often, we need to change or update a components state. One way to do this is by overriding a component's updateComponent(long timeDelta) method. This method is called each frame before the component is drawn. The method also provides the time interval (in milliseconds) that has passed since the last time the method was called. This allows us to update the state depending on the time passed. We could for example use this to change the components color over time. Another example would be a blinking text cursor which is displayed only for a certain time interval.

Relevant methods:

• updateComponent(long timeDelta)
  Allows to update the component's state before drawing it.
• setController(IMTController controller)
  This attaches a controller object to the component. The controller's update method is called everytime the component's updateComponent method is called. This allows to update the component from the outside, without extending the component and overriding its update method.