Developer Documentation
Implementing mouse and keyboard interaction

Overview

Mouse interaction (especially object picking) and keyboard shortcuts form essential and often used functionality in OpenFlipper. In this tutorial we will learn how to use picking modes, context menus and simple keyboard shortcuts. The plugin will provide the following functions:

  • Hide object by right-clicking at it and selecting our defined action
  • Select an object by entering user defined pick mode and double-clicking at object in the scene
  • Rotate selected object by hitting the j,k,h,l keys on the keyboard

For this purpose we will make use of the following Plugin Interfaces

Since we outlined the details of overriding the BaseInterface methods in the previous tutorials (A simple plugin and Implementing a mesh smoother plugin) we can directly switch over to what happens within these methods. When initializing our plugin we set the active object identifier to -1 since no object has been selected yet. We initialize the axis vectors that we'll need for the rotation later on. Then we tell OpenFlipper that we will use the j, k, h and l key on the keyboard (so it'll call slotKeyEvent() each time one of the keys has been pressed). Note: OpenFlipper will show up a warning message in the log widget if the desired keys are already assigned to another plugin or core function.

void MouseAndKeyPlugin::initializePlugin() {
// Register keys
emit registerKey(Qt::Key_J, Qt::NoModifier, "Rotate object down");
emit registerKey(Qt::Key_K, Qt::NoModifier, "Rotate object up");
emit registerKey(Qt::Key_H, Qt::NoModifier, "Rotate object left");
emit registerKey(Qt::Key_L, Qt::NoModifier, "Rotate object right");
tool_ = new QWidget();
QSize size(300, 300);
tool_->resize(size);
// Create button that can be toggled
// to (de)activate plugin's picking mode
pickButton_ = new QPushButton(tr("Select object"));
pickButton_->setCheckable(true);
// Create label
QLabel* label = new QLabel();
label->setText("(De)activate pick mode");
// Set label to be above the button
QGridLayout* grid = new QGridLayout;
grid->addWidget(label, 0, 0);
grid->addWidget(pickButton_, 1, 0);
tool_->setLayout(grid);
// Connect button to slotButtonClicked()
connect(pickButton_, SIGNAL(clicked()), this, SLOT(slotButtonClicked()));
// Add the Toolbox
emit addToolbox(tr("Mouse and Key"), tool_);
} // End initializePlugin

If all plugins have been initialized, we add our own pick mode named "MouseAndKeyPlugin" to OpenFlipper and create the context menu entry (which is actually a QMenu object containing one (or generally multiple) object(s) of type QAction) which we connect to contextMenuItemSelected().

void MouseAndKeyPlugin::pluginsInitialized() {
// Add pickmode
emit addPickMode("MouseAndKeyPlugin");
// Add context menu entry
// Create submenu
contextMenuEntry_ = new QMenu("Mouse and key plugin");
QAction* lastAction;
// Add action to recently created menu
lastAction = contextMenuEntry_->addAction( "Hide object" );
lastAction->setToolTip("Hide selected object");
lastAction->setStatusTip( lastAction->toolTip() );
// Finally insert created context submenu to OpenFlipper's context menu
// We want our action to be visible for triangle and polygon meshes
emit addContextMenuItem(contextMenuEntry_->menuAction() , DATA_TRIANGLE_MESH , CONTEXTOBJECTMENU );
emit addContextMenuItem(contextMenuEntry_->menuAction() , DATA_POLY_MESH , CONTEXTOBJECTMENU );
// Connect the created context menu entry to local function contextMenuItemSelected(QAction*)
connect(contextMenuEntry_, SIGNAL(triggered(QAction*)), this, SLOT(contextMenuItemSelected(QAction*)));
} // End pluginsInitialized

In intializeToolbox() we create a simple toolbox containing a label and a push-button. We connect the button to our method slotButtonClicked() which will then be called each time the button is clicked.

Now each time the button in our toolbox is clicked we want to activate the picking mode, such that if the button is checked pick mode "MouseAndKeyPlugin" that we defined at the beginning will be active and OpenFlipper switches over from ExamineMode to PickingMode (see PluginFunctions for details). Clicking once more at the button will return to ExamineMode (in which the user can navigate through the scene).

void MouseAndKeyPlugin::slotButtonClicked() {
if(pickButton_->isChecked()) {
// Picking mode of our plugin shall be activated
// set OpenFlipper's action mode to picking
PluginFunctions::actionMode( Viewer::PickingMode );
// Now activate our picking mode
PluginFunctions::pickMode( "MouseAndKeyPlugin" );
} else {
// Picking mode shall be deactivated
PluginFunctions::actionMode( Viewer::ExamineMode );
}
} // End slotButtonClicked

If the pick mode has been changed externally, we want our button in the toolbox to appear pressed (or unpressed respectively). _mode holds the name of the currently selected pick mode.

void MouseAndKeyPlugin::slotPickModeChanged(const std::string& _mode) {
// Set button checked if pick mode is our
// plugin's pick mode
pickButton_->setChecked(_mode == "MouseAndKeyPlugin");
} // End slotPickModeChanged

In the next method we describe how mouse actions are to be handled by the plugin. We want our plugin only to react to mouse events if our pick mode is active otherwise don't do anything. If OpenFlipper is in PickingMode and the currently active pick mode is "MouseAndKeyPlugin" we try to get the object on that the user has double clicked. If the object can be found, we'll show up a dialog window displaying the picked object's identifier and assign it to the member variable that holds the active object. Otherwise we display "Picking failed" in the log widget. Note that the mouse event has to be traversed through the rest of the scene graph after intercepting and treating the desired mouse action.

void MouseAndKeyPlugin::slotMouseEvent(QMouseEvent* _event) {
if ( PluginFunctions::pickMode() == "MouseAndKeyPlugin" &&
PluginFunctions::actionMode() == Viewer::PickingMode ) {
// If double click has been performed
if (_event->type() == QEvent::MouseButtonDblClick) {
size_t node_idx, target_idx;
OpenMesh::Vec3d hitPoint;
// Get picked object's identifier
if ( PluginFunctions::scenegraphPick(ACG::SceneGraph::PICK_ANYTHING,_event->pos(), node_idx, target_idx, &hitPoint) ) {
BaseObjectData* object;
// Get picked object
if ( PluginFunctions::getPickedObject(node_idx, object) ) {
// Show small dialog window
QDialog* dlg = new QDialog;
QGridLayout* grid = new QGridLayout;
QLabel* label = new QLabel;
QString str = QString("You selected object ");
str += QString::number(node_idx);
label->setText(str);
grid->addWidget(label, 0,0);
dlg->setLayout(grid);
dlg->show();
// Set last selected object
activeObject_ = node_idx;
}
else {
emit log(LOGINFO, "Picking failed");
}
}
return;
}
// Continue traversing scene graph
ACG::SceneGraph::MouseEventAction action(_event, PluginFunctions::viewerProperties().glState());
PluginFunctions::traverse(action);
}
} // End slotMouseEvent

Next method is called whenever any of the keys j, k, h or l is pressed. If an object has been selected (accordingly the member variable activeObject_ holds a valid objects identifier -as described before-) we try to get its handle by calling PluginFunctions::getPickedObject(). We then set the rotation matrix of the selected object's transform node (manipulatorNode) to hold a matrix that describes a rotation around the x (if j or k is pressed) or y axis (if h or l is pressed) by +/- 10 degrees. We then call the method transformMesh and pass the recently calculated matrix and a handle to the mesh (triangle or polygon). We have to inform OpenFlipper's core about the changes by emitting the updatedObject signal and specifying that the geometry has changed.

void MouseAndKeyPlugin::slotKeyEvent( QKeyEvent* _event ) {
BaseObjectData* object;
// Get last clicked object (selected in pick mode)
if ( PluginFunctions::getPickedObject(activeObject_, object) ) {
// Switch pressed keys
switch (_event->key())
{
case Qt::Key_J:
object->manipulatorNode()->loadIdentity();
object->manipulatorNode()->rotate(10.0, axis_x_);
break;
case Qt::Key_K :
object->manipulatorNode()->loadIdentity();
object->manipulatorNode()->rotate(-10.0, axis_x_);
break;
case Qt::Key_H :
object->manipulatorNode()->loadIdentity();
object->manipulatorNode()->rotate(10.0, axis_y_);
break;
case Qt::Key_L :
object->manipulatorNode()->loadIdentity();
object->manipulatorNode()->rotate(-10.0, axis_y_);
break;
default:
break;
}
// Perform rotation
if ( object->dataType( DATA_TRIANGLE_MESH ) )
transformMesh(object->manipulatorNode()->matrix(), (*PluginFunctions::triMesh(object)));
if ( object->dataType( DATA_POLY_MESH ) )
transformMesh(object->manipulatorNode()->matrix(), (*PluginFunctions::polyMesh(object)));
// Tell core that object has been modified
emit updatedObject(object->id(), UPDATE_GEOMETRY);
// Update view
emit updateView();
} else {
emit log(LOGINFO, "No object has been selected to rotate! Select object first.");
}
} // End slotKeyEvent

This template method transforms the given mesh:

Last but not least, the method that is called each time our context menu has been clicked. We get the object's id on which the user has performed a right click from the action data. Then we try to get the node and its BaseObjectData handle. If successfully passed to this point we hide the object by calling its hide() method.

void MouseAndKeyPlugin::contextMenuItemSelected(QAction* _action) {
// Get object id from QAction object
QVariant contextObject = _action->data();
int objectId = contextObject.toInt();
if (objectId == -1) {
log(LOGINFO, "Could not get associated object id.");
return;
}
// Get node associated to object id
ACG::SceneGraph::BaseNode* node = ACG::SceneGraph::find_node(
PluginFunctions::getSceneGraphRootNode(), objectId);
// Return if node id was not valid
if (!node) {
log(LOGINFO, "Could not find associated object.");
return;
}
BaseObjectData* obj;
if (!PluginFunctions::getObject(objectId, obj))
return;
// Hide object
obj->hide();
} // End contextMenuItemSelected

The complete source code of this example

We use the same project file as before.

MouseAndKeyPlugin.hh

#ifndef MOUSEANDKEYPLUGIN_HH
#define MOUSEANDKEYPLUGIN_HH
#include <OpenFlipper/BasePlugin/BaseInterface.hh>
#include <OpenFlipper/BasePlugin/MouseInterface.hh>
#include <OpenFlipper/BasePlugin/PickingInterface.hh>
#include <OpenFlipper/BasePlugin/KeyInterface.hh>
#include <OpenFlipper/BasePlugin/ContextMenuInterface.hh>
#include <OpenFlipper/BasePlugin/ToolboxInterface.hh>
#include <OpenFlipper/BasePlugin/LoggingInterface.hh>
#include <OpenFlipper/common/Types.hh>
#include <QPushButton>
#include <QLabel>
#include <QGridLayout>
#include <QMenu>
class MouseAndKeyPlugin: public QObject,
BaseInterface,
MouseInterface,
PickingInterface,
ContextMenuInterface,
ToolboxInterface,
KeyInterface,
LoggingInterface {
Q_OBJECT
Q_INTERFACES(BaseInterface)
Q_INTERFACES(MouseInterface)
Q_INTERFACES(PickingInterface)
Q_INTERFACES(KeyInterface)
Q_INTERFACES(ContextMenuInterface)
Q_INTERFACES(ToolboxInterface)
Q_INTERFACES(LoggingInterface)
Q_PLUGIN_METADATA(IID "org.OpenFlipper.Plugins.examples.MouseAndKeyPlugin")
signals:
//BaseInterface
void updateView();
void updatedObject(int _identifier, const UpdateType& _type);
//LoggingInterface
void log(Logtype _type, QString _message);
void log(QString _message);
//ContextMenuInterface
void addContextMenuItem(QAction* _action , ContextMenuType _type);
void addContextMenuItem(QAction* _action , DataType _objectType , ContextMenuType _type );
//PickingInterface
void addPickMode(const std::string& _mode);
void addHiddenPickMode(const std::string& _mode);
//KeyInterface
void registerKey(int _key, Qt::KeyboardModifiers _modifiers, QString _description, bool _multiUse = false);
// ToolboxInterface
void addToolbox(QString _name, QWidget* _widget);
private slots:
// BaseInterface
void initializePlugin();
void pluginsInitialized();
//MouseInterface
void slotMouseEvent( QMouseEvent* _event );
//KeyInterface
void slotKeyEvent( QKeyEvent* _event );
//PickingInterface
void slotPickModeChanged( const std::string& _mode);
public:
// BaseInterface
QString name() {return (QString("Mouse and Keyboard Plugin"));};
QString description() {return (QString("Shows some basic mouse and key embedding"));};
MouseAndKeyPlugin();
private:
// Transform geometry with given Matrix
template <typename MeshT>
void transformMesh(ACG::Matrix4x4d _mat , MeshT& _mesh );
// The context menu action
QMenu* contextMenuEntry_;
// The toolbox widget and the button in it
QWidget* tool_;
QPushButton* pickButton_;
// Last picked object
int activeObject_;
// Rotation axes
ACG::Vec3d axis_x_;
ACG::Vec3d axis_y_;
private slots:
// Is called each time the button in the toolbox is clicked
void slotButtonClicked();
// Is called if context menu item has been selected
void contextMenuItemSelected(QAction* _action);
public slots:
QString version() { return QString("1.0"); };
};
#endif //MOUSEANDKEYPLUGIN_HH

MouseAndKeyPlugin.cc

//=============================================================================
//
// OpenFlipper
// Copyright (C) 2008 by Computer Graphics Group, RWTH Aachen
// www.openflipper.org
//
//-----------------------------------------------------------------------------
//
// License
//
// OpenFlipper is free software: you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// OpenFlipper is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public License
// along with OpenFlipper. If not, see <http://www.gnu.org/licenses/>.
//
//-----------------------------------------------------------------------------
//
//
//=============================================================================
#include "MouseAndKeyPlugin.hh"
#include <ObjectTypes/PolyMesh/PolyMesh.hh>
#include <ObjectTypes/TriangleMesh/TriangleMesh.hh>
#include "OpenFlipper/BasePlugin/PluginFunctions.hh"
/*
* Constructor
*/
MouseAndKeyPlugin::MouseAndKeyPlugin() :
contextMenuEntry_(0),
tool_(0),
pickButton_(0),
activeObject_(-1),
axis_x_(ACG::Vec3d(1.0, 0.0, 0.0)),
axis_y_(ACG::Vec3d(0.0, 1.0, 0.0))
{
}
/*
* Initialize plugin
*/
void MouseAndKeyPlugin::initializePlugin() {
// Register keys
emit registerKey(Qt::Key_J, Qt::NoModifier, "Rotate object down");
emit registerKey(Qt::Key_K, Qt::NoModifier, "Rotate object up");
emit registerKey(Qt::Key_H, Qt::NoModifier, "Rotate object left");
emit registerKey(Qt::Key_L, Qt::NoModifier, "Rotate object right");
tool_ = new QWidget();
QSize size(300, 300);
tool_->resize(size);
// Create button that can be toggled
// to (de)activate plugin's picking mode
pickButton_ = new QPushButton(tr("Select object"));
pickButton_->setCheckable(true);
// Create label
QLabel* label = new QLabel();
label->setText("(De)activate pick mode");
// Set label to be above the button
QGridLayout* grid = new QGridLayout;
grid->addWidget(label, 0, 0);
grid->addWidget(pickButton_, 1, 0);
tool_->setLayout(grid);
// Connect button to slotButtonClicked()
connect(pickButton_, SIGNAL(clicked()), this, SLOT(slotButtonClicked()));
// Add the Toolbox
emit addToolbox(tr("Mouse and Key"), tool_);
} // End initializePlugin
/*
* Is called after all plugins have been initialized
*/
void MouseAndKeyPlugin::pluginsInitialized() {
// Add pickmode
emit addPickMode("MouseAndKeyPlugin");
// Add context menu entry
// Create submenu
contextMenuEntry_ = new QMenu("Mouse and key plugin");
QAction* lastAction;
// Add action to recently created menu
lastAction = contextMenuEntry_->addAction( "Hide object" );
lastAction->setToolTip("Hide selected object");
lastAction->setStatusTip( lastAction->toolTip() );
// Finally insert created context submenu to OpenFlipper's context menu
// We want our action to be visible for triangle and polygon meshes
emit addContextMenuItem(contextMenuEntry_->menuAction() , DATA_TRIANGLE_MESH , CONTEXTOBJECTMENU );
emit addContextMenuItem(contextMenuEntry_->menuAction() , DATA_POLY_MESH , CONTEXTOBJECTMENU );
// Connect the created context menu entry to local function contextMenuItemSelected(QAction*)
connect(contextMenuEntry_, SIGNAL(triggered(QAction*)), this, SLOT(contextMenuItemSelected(QAction*)));
} // End pluginsInitialized
/*
* Is called when button in toolbox has been clicked
*/
void MouseAndKeyPlugin::slotButtonClicked() {
if(pickButton_->isChecked()) {
// Picking mode of our plugin shall be activated
// set OpenFlipper's action mode to picking
PluginFunctions::actionMode( Viewer::PickingMode );
// Now activate our picking mode
PluginFunctions::pickMode( "MouseAndKeyPlugin" );
} else {
// Picking mode shall be deactivated
PluginFunctions::actionMode( Viewer::ExamineMode );
}
} // End slotButtonClicked
/*
* Is called when pick mode is changed in OpenFlipper
*/
void MouseAndKeyPlugin::slotPickModeChanged(const std::string& _mode) {
// Set button checked if pick mode is our
// plugin's pick mode
pickButton_->setChecked(_mode == "MouseAndKeyPlugin");
} // End slotPickModeChanged
/*
* Is called each time the mouse has moved or been clicked
*/
void MouseAndKeyPlugin::slotMouseEvent(QMouseEvent* _event) {
if ( PluginFunctions::pickMode() == "MouseAndKeyPlugin" &&
PluginFunctions::actionMode() == Viewer::PickingMode ) {
// If double click has been performed
if (_event->type() == QEvent::MouseButtonDblClick) {
size_t node_idx, target_idx;
OpenMesh::Vec3d hitPoint;
// Get picked object's identifier
if ( PluginFunctions::scenegraphPick(ACG::SceneGraph::PICK_ANYTHING,_event->pos(), node_idx, target_idx, &hitPoint) ) {
BaseObjectData* object;
// Get picked object
if ( PluginFunctions::getPickedObject(node_idx, object) ) {
// Show small dialog window
QDialog* dlg = new QDialog;
QGridLayout* grid = new QGridLayout;
QLabel* label = new QLabel;
QString str = QString("You selected object ");
str += QString::number(node_idx);
label->setText(str);
grid->addWidget(label, 0,0);
dlg->setLayout(grid);
dlg->show();
// Set last selected object
activeObject_ = node_idx;
}
else {
emit log(LOGINFO, "Picking failed");
}
}
return;
}
// Continue traversing scene graph
ACG::SceneGraph::MouseEventAction action(_event, PluginFunctions::viewerProperties().glState());
PluginFunctions::traverse(action);
}
} // End slotMouseEvent
/*
* Is called when a key on the keyboard is pressed
*/
void MouseAndKeyPlugin::slotKeyEvent( QKeyEvent* _event ) {
BaseObjectData* object;
// Get last clicked object (selected in pick mode)
if ( PluginFunctions::getPickedObject(activeObject_, object) ) {
// Switch pressed keys
switch (_event->key())
{
case Qt::Key_J:
object->manipulatorNode()->loadIdentity();
object->manipulatorNode()->rotate(10.0, axis_x_);
break;
case Qt::Key_K :
object->manipulatorNode()->loadIdentity();
object->manipulatorNode()->rotate(-10.0, axis_x_);
break;
case Qt::Key_H :
object->manipulatorNode()->loadIdentity();
object->manipulatorNode()->rotate(10.0, axis_y_);
break;
case Qt::Key_L :
object->manipulatorNode()->loadIdentity();
object->manipulatorNode()->rotate(-10.0, axis_y_);
break;
default:
break;
}
// Perform rotation
if ( object->dataType( DATA_TRIANGLE_MESH ) )
transformMesh(object->manipulatorNode()->matrix(), (*PluginFunctions::triMesh(object)));
if ( object->dataType( DATA_POLY_MESH ) )
transformMesh(object->manipulatorNode()->matrix(), (*PluginFunctions::polyMesh(object)));
// Tell core that object has been modified
emit updatedObject(object->id(), UPDATE_GEOMETRY);
// Update view
emit updateView();
} else {
emit log(LOGINFO, "No object has been selected to rotate! Select object first.");
}
} // End slotKeyEvent
/*
* Transform a mesh with the given transformation matrix
*
* _mat : transformation matrix
* _mesh : the mesh
*/
template<typename MeshT>
void MouseAndKeyPlugin::transformMesh(ACG::Matrix4x4d _mat, MeshT& _mesh) {
typename MeshT::VertexIter v_it = _mesh.vertices_begin();
typename MeshT::VertexIter v_end = _mesh.vertices_end();
// Iterator over all vertices and transform them by _mat
// Update normals
for (; v_it != v_end; ++v_it) {
_mesh.set_point(*v_it, (typename MeshT::Point) _mat.transform_point(
(OpenMesh::Vec3d)(_mesh.point(*v_it))));
_mesh.set_normal(*v_it, (typename MeshT::Point) _mat.transform_vector(
(OpenMesh::Vec3d)(_mesh.normal(*v_it))));
}
typename MeshT::FaceIter f_it = _mesh.faces_begin();
typename MeshT::FaceIter f_end = _mesh.faces_end();
// Iterate over all faces and update face normals
for (; f_it != f_end; ++f_it)
_mesh.set_normal(*f_it, (typename MeshT::Point) _mat.transform_vector(
(OpenMesh::Vec3d)(_mesh.normal(*f_it))));
} // End transformMesh
/*
* Is called when context menu entry has been clicked
*/
void MouseAndKeyPlugin::contextMenuItemSelected(QAction* _action) {
// Get object id from QAction object
QVariant contextObject = _action->data();
int objectId = contextObject.toInt();
if (objectId == -1) {
log(LOGINFO, "Could not get associated object id.");
return;
}
// Get node associated to object id
ACG::SceneGraph::BaseNode* node = ACG::SceneGraph::find_node(
PluginFunctions::getSceneGraphRootNode(), objectId);
// Return if node id was not valid
if (!node) {
log(LOGINFO, "Could not find associated object.");
return;
}
BaseObjectData* obj;
if (!PluginFunctions::getObject(objectId, obj))
return;
// Hide object
obj->hide();
} // End contextMenuItemSelected