Implementing mouse and keyboard interaction

Overview

Mouse interaction (especially object picking) and keyboard shortcuts form essential and often used functionality in OpenFlipper. In this tutorial we will learn how to use picking modes, context menus and simple keyboard shortcuts. The plugin will provide the following functions:

• Hide object by right-clicking at it and selecting our defined action
• Select an object by entering user defined pick mode and double-clicking at object in the scene
• Rotate selected object by hitting the j,k,h,l keys on the keyboard

For this purpose we will make use of the following Plugin Interfaces

• Base Interface
• Mouse Interface
• Picking Interface/Mouse Picking/Pick Mode Toolbars
• Key Interface
• Context Menu Interface
• Toolbox Interface
• Logging Interface

Since we outlined the details of overriding the BaseInterface methods in the previous tutorials (A simple plugin and Implementing a mesh smoother plugin) we can directly switch over to what happens within these methods. When initializing our plugin we set the active object identifier to -1 since no object has been selected yet. We initialize the axis vectors that we'll need for the rotation later on. Then we tell OpenFlipper that we will use the j, k, h and l key on the keyboard (so it'll call slotKeyEvent() each time one of the keys has been pressed). Note: OpenFlipper will show up a warning message in the log widget if the desired keys are already assigned to another plugin or core function.

void MouseAndKeyPlugin::initializePlugin() {

        // Register keys
        emit registerKey(Qt::Key_J,     Qt::NoModifier, "Rotate object down");
        emit registerKey(Qt::Key_K,     Qt::NoModifier, "Rotate object up");
        emit registerKey(Qt::Key_H,     Qt::NoModifier, "Rotate object left");
        emit registerKey(Qt::Key_L,     Qt::NoModifier, "Rotate object right");

  tool_ = new QWidget();
  QSize size(300, 300);
  tool_->resize(size);

  // Create button that can be toggled
  // to (de)activate plugin's picking mode
  pickButton_ = new QPushButton(tr("Select object"));
  pickButton_->setCheckable(true);

  // Create label
  QLabel* label = new QLabel();
  label->setText("(De)activate pick mode");

  // Set label to be above the button
  QGridLayout* grid = new QGridLayout;
  grid->addWidget(label, 0, 0);
  grid->addWidget(pickButton_, 1, 0);

  tool_->setLayout(grid);

  // Connect button to slotButtonClicked()
  connect(pickButton_, SIGNAL(clicked()), this, SLOT(slotButtonClicked()));

  // Add the Toolbox
  emit addToolbox(tr("Mouse and Key"), tool_);

} // End initializePlugin

If all plugins have been initialized, we add our own pick mode named "MouseAndKeyPlugin" to OpenFlipper and create the context menu entry (which is actually a QMenu object containing one (or generally multiple) object(s) of type QAction) which we connect to contextMenuItemSelected().

void MouseAndKeyPlugin::pluginsInitialized() {

        // Add pickmode
        emit addPickMode("MouseAndKeyPlugin");

        // Add context menu entry

        // Create submenu
        contextMenuEntry_ = new QMenu("Mouse and key plugin");

        QAction* lastAction;

        // Add action to recently created menu
        lastAction = contextMenuEntry_->addAction( "Hide object" );
        lastAction->setToolTip("Hide selected object");
        lastAction->setStatusTip( lastAction->toolTip() );

        // Finally insert created context submenu to OpenFlipper's context menu
        // We want our action to be visible for triangle and polygon meshes
        emit addContextMenuItem(contextMenuEntry_->menuAction() , DATA_TRIANGLE_MESH , CONTEXTOBJECTMENU );
        emit addContextMenuItem(contextMenuEntry_->menuAction() , DATA_POLY_MESH , CONTEXTOBJECTMENU );

        // Connect the created context menu entry to local function contextMenuItemSelected(QAction*)
        connect(contextMenuEntry_, SIGNAL(triggered(QAction*)), this, SLOT(contextMenuItemSelected(QAction*)));

} // End pluginsInitialized

In intializeToolbox() we create a simple toolbox containing a label and a push-button. We connect the button to our method slotButtonClicked() which will then be called each time the button is clicked.

Now each time the button in our toolbox is clicked we want to activate the picking mode, such that if the button is checked pick mode "MouseAndKeyPlugin" that we defined at the beginning will be active and OpenFlipper switches over from ExamineMode to PickingMode (see PluginFunctions for details). Clicking once more at the button will return to ExamineMode (in which the user can navigate through the scene).

void MouseAndKeyPlugin::slotButtonClicked() {

        if(pickButton_->isChecked()) {
                // Picking mode of our plugin shall be activated
                // set OpenFlipper's action mode to picking
                PluginFunctions::actionMode( Viewer::PickingMode );

                // Now activate our picking mode
                PluginFunctions::pickMode( "MouseAndKeyPlugin" );
        } else {
                // Picking mode shall be deactivated
                PluginFunctions::actionMode( Viewer::ExamineMode );
        }

} // End slotButtonClicked

If the pick mode has been changed externally, we want our button in the toolbox to appear pressed (or unpressed respectively). _mode holds the name of the currently selected pick mode.

void MouseAndKeyPlugin::slotPickModeChanged(const std::string& _mode) {

        // Set button checked if pick mode is our
        // plugin's pick mode
        pickButton_->setChecked(_mode == "MouseAndKeyPlugin");

} // End slotPickModeChanged

In the next method we describe how mouse actions are to be handled by the plugin. We want our plugin only to react to mouse events if our pick mode is active otherwise don't do anything. If OpenFlipper is in PickingMode and the currently active pick mode is "MouseAndKeyPlugin" we try to get the object on that the user has double clicked. If the object can be found, we'll show up a dialog window displaying the picked object's identifier and assign it to the member variable that holds the active object. Otherwise we display "Picking failed" in the log widget. Note that the mouse event has to be traversed through the rest of the scene graph after intercepting and treating the desired mouse action.

void MouseAndKeyPlugin::slotMouseEvent(QMouseEvent* _event) {

        if ( PluginFunctions::pickMode() == "MouseAndKeyPlugin" &&
                PluginFunctions::actionMode() == Viewer::PickingMode ) {

                // If double click has been performed
                if (_event->type() == QEvent::MouseButtonDblClick) {

                        unsigned int node_idx, target_idx;
                        OpenMesh::Vec3d hitPoint;

                        // Get picked object's identifier
                        if ( PluginFunctions::scenegraphPick(ACG::SceneGraph::PICK_ANYTHING,_event->pos(), node_idx, target_idx, &hitPoint) ) {

                                BaseObjectData* object;

                                // Get picked object
                                if ( PluginFunctions::getPickedObject(node_idx, object) ) {

                                        // Show small dialog window
                                        QDialog* dlg = new QDialog;

                                        QGridLayout* grid = new QGridLayout;
                                        QLabel* label = new QLabel;
                                        QString str = QString("You selected object ");
                                        str += QString::number(node_idx);
                                        label->setText(str);
                                        grid->addWidget(label, 0,0);

                                        dlg->setLayout(grid);

                                        dlg->show();

                                        // Set last selected object
                                        activeObject_ = node_idx;
                                }
                                else {
                                        emit log(LOGINFO, "Picking failed");
                                }
                        }

                        return;
                }

                // Continue traversing scene graph
                ACG::SceneGraph::MouseEventAction action(_event, PluginFunctions::viewerProperties().glState());
                PluginFunctions::traverse(action);
        }

} // End slotMouseEvent

Next method is called whenever any of the keys j, k, h or l is pressed. If an object has been selected (accordingly the member variable activeObject_ holds a valid objects identifier -as described before-) we try to get its handle by calling PluginFunctions::getPickedObject(). We then set the rotation matrix of the selected object's transform node (manipulatorNode) to hold a matrix that describes a rotation around the x (if j or k is pressed) or y axis (if h or l is pressed) by +/- 10 degrees. We then call the method transformMesh and pass the recently calculated matrix and a handle to the mesh (triangle or polygon). We have to inform OpenFlipper's core about the changes by emitting the updatedObject signal and specifying that the geometry has changed.

void MouseAndKeyPlugin::slotKeyEvent( QKeyEvent* _event ) {

        BaseObjectData* object;

        // Get last clicked object (selected in pick mode)
        if ( PluginFunctions::getPickedObject(activeObject_, object) ) {

                // Switch pressed keys
                switch (_event->key())
                {
                        case Qt::Key_J:

                                object->manipulatorNode()->loadIdentity();
                                object->manipulatorNode()->rotate(10.0, axis_x_);

                                break;

                        case Qt::Key_K :

                                object->manipulatorNode()->loadIdentity();
                                object->manipulatorNode()->rotate(-10.0, axis_x_);

                                break;

                        case Qt::Key_H :

                                object->manipulatorNode()->loadIdentity();
                                object->manipulatorNode()->rotate(10.0, axis_y_);

                                break;

                        case Qt::Key_L :

                                object->manipulatorNode()->loadIdentity();
                                object->manipulatorNode()->rotate(-10.0, axis_y_);

                                break;

                        default:
                        break;
                }

                // Perform rotation
                if ( object->dataType( DATA_TRIANGLE_MESH ) )
                        transformMesh(object->manipulatorNode()->matrix(), (*PluginFunctions::triMesh(object)));
                if ( object->dataType( DATA_POLY_MESH ) )
                        transformMesh(object->manipulatorNode()->matrix(), (*PluginFunctions::polyMesh(object)));

                // Tell core that object has been modified
                emit updatedObject(object->id(), UPDATE_GEOMETRY);

                // Update view
                emit updateView();
        } else {
                emit log(LOGINFO, "No object has been selected to rotate! Select object first.");
        }

} // End slotKeyEvent

This template method transforms the given mesh:

Last but not least, the method that is called each time our context menu has been clicked. We get the object's id on which the user has performed a right click from the action data. Then we try to get the node and its BaseObjectData handle. If successfully passed to this point we hide the object by calling its hide() method.

void MouseAndKeyPlugin::contextMenuItemSelected(QAction* _action) {

        // Get object id from QAction object
        QVariant contextObject = _action->data();
        int objectId = contextObject.toInt();

        if (objectId == -1) {

                log(LOGINFO, "Could not get associated object id.");
                return;
        }

        // Get node associated to object id
        ACG::SceneGraph::BaseNode* node = ACG::SceneGraph::find_node(
                        PluginFunctions::getSceneGraphRootNode(), objectId);

        // Return if node id was not valid
        if (!node) {

                log(LOGINFO, "Could not find associated object.");
                return;
        }

        BaseObjectData* obj;
        if (!PluginFunctions::getObject(objectId, obj))
                return;

        // Hide object
        obj->hide();

} // End contextMenuItemSelected

The complete source code of this example

We use the same project file as before.

MouseAndKeyPlugin.hh

#ifndef MOUSEANDKEYPLUGIN_HH
#define MOUSEANDKEYPLUGIN_HH

#include <OpenFlipper/BasePlugin/BaseInterface.hh>
#include <OpenFlipper/BasePlugin/MouseInterface.hh>
#include <OpenFlipper/BasePlugin/PickingInterface.hh>
#include <OpenFlipper/BasePlugin/KeyInterface.hh>
#include <OpenFlipper/BasePlugin/ContextMenuInterface.hh>
#include <OpenFlipper/BasePlugin/ToolboxInterface.hh>
#include <OpenFlipper/BasePlugin/LoggingInterface.hh>
#include <OpenFlipper/common/Types.hh>

class MouseAndKeyPlugin: public QObject,
                BaseInterface,
                MouseInterface,
                PickingInterface,
                ContextMenuInterface,
                ToolboxInterface,
                KeyInterface,
                LoggingInterface {
        Q_OBJECT
        Q_INTERFACES(BaseInterface)
        Q_INTERFACES(MouseInterface)
        Q_INTERFACES(PickingInterface)
        Q_INTERFACES(KeyInterface)
        Q_INTERFACES(ContextMenuInterface)
        Q_INTERFACES(ToolboxInterface)
        Q_INTERFACES(LoggingInterface)

        signals:

        //BaseInterface
        void updateView();
        void updatedObject(int _identifier, const UpdateType& _type);
        //LoggingInterface
        void log(Logtype _type, QString _message);
        void log(QString _message);
        //ContextMenuInterface
        void addContextMenuItem(QAction* _action , ContextMenuType _type);
        void addContextMenuItem(QAction* _action , DataType _objectType , ContextMenuType _type );
        //PickingInterface
        void addPickMode(const std::string& _mode);
        void addHiddenPickMode(const std::string& _mode);
        //KeyInterface
        void registerKey(int _key, Qt::KeyboardModifiers _modifiers, QString _description, bool _multiUse = false);

  // ToolboxInterface
  void addToolbox(QString _name, QWidget* _widget);

        private slots:

        // BaseInterface
        void initializePlugin();
        void pluginsInitialized();
        //MouseInterface
        void slotMouseEvent( QMouseEvent* _event );
        //KeyInterface
        void slotKeyEvent( QKeyEvent* _event );
        //PickingInterface
        void slotPickModeChanged( const std::string& _mode);

        public:

        // BaseInterface
        QString name() {return (QString("Mouse and Keyboard Plugin"));};
        QString description() {return (QString("Shows some basic mouse and key embedding"));};

        MouseAndKeyPlugin();

        private:

        // Transform geometry with given Matrix
        template <typename MeshT>
        void transformMesh(ACG::Matrix4x4d _mat , MeshT& _mesh );

        // The context menu action
        QMenu* contextMenuEntry_;

        // The toolbox widget and the button in it
        QWidget* tool_;
        QPushButton* pickButton_;

        // Last picked object
        int activeObject_;

        // Rotation axes
        ACG::Vec3d axis_x_;
        ACG::Vec3d axis_y_;

        private slots:

        // Is called each time the button in the toolbox is clicked
        void slotButtonClicked();

        // Is called if context menu item has been selected
        void contextMenuItemSelected(QAction* _action);

        public slots:
        QString version() { return QString("1.0"); };
};

#endif //MOUSEANDKEYPLUGIN_HH

MouseAndKeyPlugin.cc

//=============================================================================
//
//                               OpenFlipper
//        Copyright (C) 2008 by Computer Graphics Group, RWTH Aachen
//                           www.openflipper.org
//
//-----------------------------------------------------------------------------
//
//                                License
//
//  OpenFlipper is free software: you can redistribute it and/or modify
//  it under the terms of the GNU Lesser General Public License as published by
//  the Free Software Foundation, either version 3 of the License, or
//  (at your option) any later version.
//
//  OpenFlipper is distributed in the hope that it will be useful,
//  but WITHOUT ANY WARRANTY; without even the implied warranty of
//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
//  GNU Lesser General Public License for more details.
//
//  You should have received a copy of the GNU Lesser General Public License
//  along with OpenFlipper.  If not, see <http://www.gnu.org/licenses/>.
//
//-----------------------------------------------------------------------------
//
//   $Revision$
//   $Author$
//   $Date$
//
//=============================================================================

#include "MouseAndKeyPlugin.hh"


#include <ObjectTypes/PolyMesh/PolyMesh.hh>
#include <ObjectTypes/TriangleMesh/TriangleMesh.hh>

#include "OpenFlipper/BasePlugin/PluginFunctions.hh"

/*
 * Constructor
 */
MouseAndKeyPlugin::MouseAndKeyPlugin() :
        contextMenuEntry_(0),
        tool_(0),
        pickButton_(0),
        activeObject_(-1),
        axis_x_(ACG::Vec3d(1.0, 0.0, 0.0)),
        axis_y_(ACG::Vec3d(0.0, 1.0, 0.0))
{

}

/*
 * Initialize plugin
 */
void MouseAndKeyPlugin::initializePlugin() {

        // Register keys
        emit registerKey(Qt::Key_J,     Qt::NoModifier, "Rotate object down");
        emit registerKey(Qt::Key_K,     Qt::NoModifier, "Rotate object up");
        emit registerKey(Qt::Key_H,     Qt::NoModifier, "Rotate object left");
        emit registerKey(Qt::Key_L,     Qt::NoModifier, "Rotate object right");

  tool_ = new QWidget();
  QSize size(300, 300);
  tool_->resize(size);

  // Create button that can be toggled
  // to (de)activate plugin's picking mode
  pickButton_ = new QPushButton(tr("Select object"));
  pickButton_->setCheckable(true);

  // Create label
  QLabel* label = new QLabel();
  label->setText("(De)activate pick mode");

  // Set label to be above the button
  QGridLayout* grid = new QGridLayout;
  grid->addWidget(label, 0, 0);
  grid->addWidget(pickButton_, 1, 0);

  tool_->setLayout(grid);

  // Connect button to slotButtonClicked()
  connect(pickButton_, SIGNAL(clicked()), this, SLOT(slotButtonClicked()));

  // Add the Toolbox
  emit addToolbox(tr("Mouse and Key"), tool_);

} // End initializePlugin

/*
 * Is called after all plugins have been initialized
 */
void MouseAndKeyPlugin::pluginsInitialized() {

        // Add pickmode
        emit addPickMode("MouseAndKeyPlugin");

        // Add context menu entry

        // Create submenu
        contextMenuEntry_ = new QMenu("Mouse and key plugin");

        QAction* lastAction;

        // Add action to recently created menu
        lastAction = contextMenuEntry_->addAction( "Hide object" );
        lastAction->setToolTip("Hide selected object");
        lastAction->setStatusTip( lastAction->toolTip() );

        // Finally insert created context submenu to OpenFlipper's context menu
        // We want our action to be visible for triangle and polygon meshes
        emit addContextMenuItem(contextMenuEntry_->menuAction() , DATA_TRIANGLE_MESH , CONTEXTOBJECTMENU );
        emit addContextMenuItem(contextMenuEntry_->menuAction() , DATA_POLY_MESH , CONTEXTOBJECTMENU );

        // Connect the created context menu entry to local function contextMenuItemSelected(QAction*)
        connect(contextMenuEntry_, SIGNAL(triggered(QAction*)), this, SLOT(contextMenuItemSelected(QAction*)));

} // End pluginsInitialized


/*
 * Is called when button in toolbox has been clicked
 */
void MouseAndKeyPlugin::slotButtonClicked() {

        if(pickButton_->isChecked()) {
                // Picking mode of our plugin shall be activated
                // set OpenFlipper's action mode to picking
                PluginFunctions::actionMode( Viewer::PickingMode );

                // Now activate our picking mode
                PluginFunctions::pickMode( "MouseAndKeyPlugin" );
        } else {
                // Picking mode shall be deactivated
                PluginFunctions::actionMode( Viewer::ExamineMode );
        }

} // End slotButtonClicked

/*
 * Is called when pick mode is changed in OpenFlipper
 */
void MouseAndKeyPlugin::slotPickModeChanged(const std::string& _mode) {

        // Set button checked if pick mode is our
        // plugin's pick mode
        pickButton_->setChecked(_mode == "MouseAndKeyPlugin");

} // End slotPickModeChanged

/*
 * Is called each time the mouse has moved or been clicked
 */
void MouseAndKeyPlugin::slotMouseEvent(QMouseEvent* _event) {

        if ( PluginFunctions::pickMode() == "MouseAndKeyPlugin" &&
                PluginFunctions::actionMode() == Viewer::PickingMode ) {

                // If double click has been performed
                if (_event->type() == QEvent::MouseButtonDblClick) {

                        unsigned int node_idx, target_idx;
                        OpenMesh::Vec3d hitPoint;

                        // Get picked object's identifier
                        if ( PluginFunctions::scenegraphPick(ACG::SceneGraph::PICK_ANYTHING,_event->pos(), node_idx, target_idx, &hitPoint) ) {

                                BaseObjectData* object;

                                // Get picked object
                                if ( PluginFunctions::getPickedObject(node_idx, object) ) {

                                        // Show small dialog window
                                        QDialog* dlg = new QDialog;

                                        QGridLayout* grid = new QGridLayout;
                                        QLabel* label = new QLabel;
                                        QString str = QString("You selected object ");
                                        str += QString::number(node_idx);
                                        label->setText(str);
                                        grid->addWidget(label, 0,0);

                                        dlg->setLayout(grid);

                                        dlg->show();

                                        // Set last selected object
                                        activeObject_ = node_idx;
                                }
                                else {
                                        emit log(LOGINFO, "Picking failed");
                                }
                        }

                        return;
                }

                // Continue traversing scene graph
                ACG::SceneGraph::MouseEventAction action(_event, PluginFunctions::viewerProperties().glState());
                PluginFunctions::traverse(action);
        }

} // End slotMouseEvent

/*
 * Is called when a key on the keyboard is pressed
 */
void MouseAndKeyPlugin::slotKeyEvent( QKeyEvent* _event ) {

        BaseObjectData* object;

        // Get last clicked object (selected in pick mode)
        if ( PluginFunctions::getPickedObject(activeObject_, object) ) {

                // Switch pressed keys
                switch (_event->key())
                {
                        case Qt::Key_J:

                                object->manipulatorNode()->loadIdentity();
                                object->manipulatorNode()->rotate(10.0, axis_x_);

                                break;

                        case Qt::Key_K :

                                object->manipulatorNode()->loadIdentity();
                                object->manipulatorNode()->rotate(-10.0, axis_x_);

                                break;

                        case Qt::Key_H :

                                object->manipulatorNode()->loadIdentity();
                                object->manipulatorNode()->rotate(10.0, axis_y_);

                                break;

                        case Qt::Key_L :

                                object->manipulatorNode()->loadIdentity();
                                object->manipulatorNode()->rotate(-10.0, axis_y_);

                                break;

                        default:
                        break;
                }

                // Perform rotation
                if ( object->dataType( DATA_TRIANGLE_MESH ) )
                        transformMesh(object->manipulatorNode()->matrix(), (*PluginFunctions::triMesh(object)));
                if ( object->dataType( DATA_POLY_MESH ) )
                        transformMesh(object->manipulatorNode()->matrix(), (*PluginFunctions::polyMesh(object)));

                // Tell core that object has been modified
                emit updatedObject(object->id(), UPDATE_GEOMETRY);

                // Update view
                emit updateView();
        } else {
                emit log(LOGINFO, "No object has been selected to rotate! Select object first.");
        }

} // End slotKeyEvent

/*
 * Transform a mesh with the given transformation matrix
 *
 * _mat : transformation matrix
 * _mesh : the mesh
 */
template<typename MeshT>
void MouseAndKeyPlugin::transformMesh(ACG::Matrix4x4d _mat, MeshT& _mesh) {

        typename MeshT::VertexIter v_it = _mesh.vertices_begin();
        typename MeshT::VertexIter v_end = _mesh.vertices_end();

        // Iterator over all vertices and transform them by _mat
        // Update normals
        for (; v_it != v_end; ++v_it) {
                _mesh.set_point(v_it, (typename MeshT::Point) _mat.transform_point(
                                (OpenMesh::Vec3d)(_mesh.point(v_it))));
                _mesh.set_normal(v_it, (typename MeshT::Point) _mat.transform_vector(
                                (OpenMesh::Vec3d)(_mesh.normal(v_it))));
        }

        typename MeshT::FaceIter f_it = _mesh.faces_begin();
        typename MeshT::FaceIter f_end = _mesh.faces_end();

        // Iterate over all faces and update face normals
        for (; f_it != f_end; ++f_it)
                _mesh.set_normal(f_it, (typename MeshT::Point) _mat.transform_vector(
                                (OpenMesh::Vec3d)(_mesh.normal(f_it))));

} // End transformMesh

/*
 * Is called when context menu entry has been clicked
 */
void MouseAndKeyPlugin::contextMenuItemSelected(QAction* _action) {

        // Get object id from QAction object
        QVariant contextObject = _action->data();
        int objectId = contextObject.toInt();

        if (objectId == -1) {

                log(LOGINFO, "Could not get associated object id.");
                return;
        }

        // Get node associated to object id
        ACG::SceneGraph::BaseNode* node = ACG::SceneGraph::find_node(
                        PluginFunctions::getSceneGraphRootNode(), objectId);

        // Return if node id was not valid
        if (!node) {

                log(LOGINFO, "Could not find associated object.");
                return;
        }

        BaseObjectData* obj;
        if (!PluginFunctions::getObject(objectId, obj))
                return;

        // Hide object
        obj->hide();

} // End contextMenuItemSelected

Q_EXPORT_PLUGIN2( mouseandkeyplugin , MouseAndKeyPlugin );