Empirical guidelines for forest management decision support systems based on the past experiences of the experts community

Aim of the study: Decision support systems for forest management (FMDSS) have been developed world wide to account for a broad range of forest ecosystems, management goals and organizational frameworks (e.g. the wiki page of the FORSYS project reports 62 existing FMDSSs from 23 countries). The need to enhance the collaboration among this diverse community of developers and users fostered the rise of new group communication processes that could capture useful knowledge from past experiences in order to efficiently provide it to new FMDSS development efforts. Material and methods: This paper presents and tests an exploratory process aiming to identify the empirical guidelines assisting developers and users of FMDSS. This process encompasses a Delphi survey built upon the consolidation of the lessons-learned statements that summarize the past experiences of the experts involved in the FORSYS project. The experts come from 34 countries and have diverse interests, ranging from forest planners, IT developers, social scientists studying participatory planning, and researchers with interests in knowledge management and in quantitative models for forest planning. Main results: The proposed 37 empirical guidelines that group 102 lessons-learned cover a broad range of issues including the DSS development cycle, involvement of the stakeholders, methods, models and knowledgebased techniques in use. Research highlights: These results may be used for improving new FMDSS development processes, teaching and training and further suggest new features of FMDSS and future research topics. Furthermore, the guidelines may constitute a knowledge repository that may be continuously improved by a community of practice.


Introduction
Forests serve a multitude of functions, including the provision of timber and non-timber forest products, clean water, carbon storage, recreation, and biodiver-sity. Major European policy initiatives, such as the Ministerial Conference on the Protection of Forests in Europe (http://www.foresteurope.org) and EU Strategic Research Agenda for the Forest-Based Sector (EU 2010) are being implemented to support multifunctional forest management. Addressing these diverse goals to satisfy the needs of forest owners, the forest industry, and society poses a considerable challenge for forest managers. A number of computer-based systems that help analyse and display forest data have been developed to help managers with the complexity of forest planning (e.g. Reynolds et al., 2008). Yet, the need for coordination in the development and application of the forest management decision support systems (FMDSS) motivated the establishment of the European network for forest decision support (FORSYS) as a project of EU-COST. The researchers and users of the FORSYS community focus on organizing and sharing the knowledge on FMDSS in Europe. For that purpose, a FORSYS Semantic Wiki Web was built and currently describes 62 existing FMDSSs from 23 countries. The emphasis is on the architecture of these systems, the models & methods used to support decision-making, the knowledge management tools and participatory processes adopted by the stakeholders engaged on forest management. The FORSYS community further aims at defining guidelines for future work on FMDSS. The FORSYS guidelines are statements or other indications of policy or procedure, aiming to assist developers and users on appropriate courses of action for the successful development of FMDSS.
This article reports the initial efforts towards the development of guidelines for FMDSS. The definition and use of guidelines are uncommon in the field of natural resource management. Nevertheless, guidelines have been successfully used to assist practitioners in information systems design, medicine and health care services. The Programming Style Guidelines firstly proposed by Kernighan and Plauger (1978) and the Human Interface Guidelines (e.g. GNOME, 2012; Android, 2012) are examples of recommendations or best-practices used by systems developers to enhance the intuitiveness, consistency and maintainability of the source code and the application interfaces, respectively. The practice guidelines aim to assist the practitioners and patients decisions on a wide variety of topics (e.g. health promotion, screening, diagnosis) and may further play an important role in health policy formation (Grilli et al., 2000;Woolf et al., 1999).
The guidelines and the instructions about guidelines utilization are often freely available at dedicated web portals (e.g. G-I-N, 2012;SIGN, 2012). However, the process used to define the guidelines is often undocumented, loosely structured and specific of each working domain. It may rely on consensus among experts during periodic meetings that take place in the course of large-scale collaborative projects (often in informa-tion systems). Or, guidelines may be the outcome of individual initiatives from a large number of experts that combine the literature review with their working experience (often in medicine and health care). In the latter, appropriate rating methods are implemented and used by a broader community of experts that are asked to assess the validity and reliability of the proposed practice guidelines. The AGREE II (Agree, 2009) is an example of such on-line rating tools in use to assess the quality of the practice guidelines.
Newly group communication processes for defining guidelines may be key for assuring the consistency and validity of the arising empirical guidelines. These processes may rely on techniques for capturing tacit knowledge and past experiences of a community of experts as the basis for the guidelines identification.
The most common approaches used in the field of forest management to structure and share knowledge based on experience include case studies, literature reviews and surveys. In particular, case studies are widely used for synthesizing past experiences in the field of information systems. The term does not refer to a single process or method, but rather the application of a variety of methods (personal experience, interviews, document research, etc.) to one or more particular instances of a phenomenon, such as an organization, event, or initiative (Yin, 2003). In relation to FMDSS, the case is typically the use of a system to address specific forest planning problems. Most of the FMDSS literature focuses on the system architecture and/or novel models and methods developed for a forest management problem and often include hypothetical cases designed to illustrate functionality (e.g. Pretzsch et al., 2006). Few real-world situations have been reported. Such case studies have the advantage of enabling detailed understanding of both the tools and the context in which they are applied. On the other hand, this focus tends to make case studies idiosyncratic. The def inition of broader case studies (e.g. Bailey, 1986;Barber and Rodman, 1990;Cortner and Schweitzer, 1983;Iverson and Alston, 1986;Johnson, 1987;Kent et al., 1991) or the comparison among multiple case studies (Gordon, 2006;Johnson et al., 2007) may provide valuable information for establishing common architecture features and general development processes for FMDSS adequate to specific forest planning problems. Yet, this method often fails to capture the tacit knowledge of the experts engaged in the development processes as well as their experiences in unsuccessful developments.
The few literature reviews in this f ield of study include single journal articles (Mendoza and Vanclay, 2008), chapters in larger works addressing DSS in general (Reynolds et al., 2008) and ecosystem management in particular (Oliver and Twery, 2000;Rauscher, 1999;Reynolds et al., 2000). More recent themes around which reviews have been structured include sustainable forestry (Rauscher et al., 2006;Reynolds, 2005;Shao and Reynolds, 2006), simulation models (Landsberg, 2003;Mäkelä et al., 2000;Muys et al., 2010;Pretzsch et al., 2008) and the use of multi-criteria decision analysis approaches (Díaz-Balteiro and Romero, 2008). These reviews provide easy information access and have a broad coverage of a topic area. They may compare existing FMDSS, discuss common features and highlight gaps between existing systems and theoretical user needs. However, this method relies exclusively on formal literature. This is a disadvantage especially in an applied field such as FMDSS, where much of the experience may not be documented in this academic fashion. Additionally, in this synthesized form, it can be difficult to tell source and quality of the information in the individual studies included. The uncoordinated nature of included studies means coverage of subtopics and their consistency are likely to be uneven. In all of the above-mentioned reviews, except one (Díaz-Balteiro and Romero, 2008), the selection process is undocumented (assumed to be the personal knowledge of the authors). These reviews also vary in terms of attention to identifying specific guidelines for future FMDSS developments versus simply summarizing existing research.
The surveys published in this field mainly focus on the inventory of the existing FMDSS (Johnson et al., 2007;Lee et al., 2003;Mowrer, 1997;Schuster et al., 1993). These studies have drawn their lessons from a comparison of the capabilities of existing systems to a synthesis of needs based on theoretical definitions of the problem from the literature. They fail to capture tacit knowledge from the experts' community. Fürst et al. (2010) is the only example of a surveytype approach with emphasis on the capturing tacit requirements from the potential users. In two workshops, they used a mind-mapping technique and a Delphi survey to attain future application areas for forest management support tools and users' requirements or desirable system properties, respectively. Nonetheless, no guidelines were drawn from this process.
Other exploratory studies have been using the Delphi survey technique to capture tacit knowledge and attain consensus from a group of experts when limited evidence exists on the specif ic topic in question (e.g. Okoli and Pawlowski, 2004). Nevo and Chan (2007) provide a recent example with similar objectives to our own; they synthesized success factors in the design and application of knowledge management systems. Jaana et al. (2011) also used the Delphi technique to develop a consensual list of key information and technology issues faced by the directors of public hospitals.
These studies suggest that the Delphi survey approaches may be adequate to capture unpublished experiences that would be omitted in a literature review and to cover these experiences in a broader, more synthesized fashion than possible with case studies.
This article extends the work of Fürst et al. (2010) and applies a new modified Delphi survey to approach the lessons-learned by the FORSYS community from both successful and failure practices.

Material and methods
The process of guidelines definition was based on a new modified Delphi survey that captured empirical evidence of the FORSYS experts. In general terms, the Delphi technique relies on repeated responses of questionnaires and controlled feedback to obtain consensus from a group of experts. It generally involves relatively small groups of experts asked to anonymously provide written answers to a set of questions over two or more rounds, with the opportunity to revise their answers based on the input of others. It is assumed that Delphi technique takes advantage of the power of groups to make better decisions than individuals on average (Okoli and Pawlowski, 2004).
There are a lot of variations of the original Delphi technique (Brown, 1968), but in general the variations of Delphi methods can be summarised as requiring (Okoli and Pawlowski, 2004): a) some feedback of individual contributions of information and knowledge; b) some assessment of the group judgment or views; c) some opportunity for individuals to revise views and; d) some degree of anonymity for the individual responses. This research extended one of the earliest definitions of expertness in the Delphi studies (Brown, 1968), which addressed the status among the peers, number of years of professional experience, or the combination of objective indicators of expertness and a priori judgement factors. This study further considered the nature of experts' knowledge and its complexity, defining the expert as someone who has either explicit or tacit knowledge about a specif ic domain of FMDSS development or use. As a consequence, the empirical guidelines evolved from the experience-based knowledge of panellists, who communicated their lessons-learned within the Delphi framework rather than separated their normative knowledge from the tacit knowledge. In addition, the developer perspective was not separated from the user perspective.
As the Delphi method relies on group dynamics for arriving at consensus rather than statistical power, a panel of 10-18 experts has been seen as sufficient number of respondents (Okoli and Pawlowski, 2004).
The proposed process for guidelines definition encompassed two rounds (Fig. 1).
The first exploratory round aimed to capture the experts' past experiences synthesized as lessonslearned, while the second aimed to rank and cluster them into the empirical guidelines. Both rounds were coordinated by the guideline development group, composed by the experts that represented the several FORSYS Work Groups, including DSS Architecture (WG1), Models and Methods (WG2), Knowledge Management (WG3) and Participatory Processes (WG4). The panellists were further classif ied into 4 main profiles. The academics primarily focused on lecturing, the researchers within a university framework, the developers engaged in DSS developments for commer-cial purposes and the government scientists working on governmental institutions.

First round of the Delphi survey
The first round of the Delphi survey involved the international panel of experts participating in the FORSYS workshop held in Thessaloniki in the 6 th of May 2011. The panel consisted of 29 experts from 18 nationalities and various domains of FMDSS application and types of organizations ( Table 1). The percentage of panellists interested on Methods and Models and Knowledge Management techniques resembled the percentage of members of the WG2 and WG3 (around 35% and 20% respectively). Yet, the percentage of panellists interested on DSS Architecture and Design was bigger than the percentage of members of WG1 (31% against 18%). The percentage of panellists from the Participatory Planning working group (WG4) was lower than the percentage of members of WG4 (around 10% against 20%).
This first round of the Delphi survey had 3 main phases (Fig. 1). During the first phase, the panellists were asked to brainstorm on their personal experiences with the development and use of FMDSS. For this purpose, the panellists were firstly asked to frame both their successful and failure practices into five major domains of DSS Development and Use, namely DSS Empirical guideliness for FMDSS based on past experiences 323

Brainstorm
Panellist perform brainstorm for personal experiences (success and failure)

Clustering
Judges cluster the lessons-learned into groups (empirical guidelines) and design the questionnaire

Rank & assessment
Panellist rank all the empirical guidelines and assess the lessonslearned statements In the second phase of this round, the guideline development group encouraged pair-wise group communication to consolidate the individual past experiences into common successful and failure practices in FMDSS development and use. Most of the Delphi studies (e.g. Nevo and Chan, 2007;Jaana et al., 2011) did not address failure practices nor included the pairwise debate. The consolidation phase conducted by the panellists may speed the process of attaining consensus among the panellists and further facilitate the work of the guideline development group. For this purpose, the individuals were grouped in pairs and the pairs reached decisions through a process of informal consensus; each individual in the pair was able to communicate his practices, and the consolidated evaluation was recorded on the open-ended questionnaire and submitted to the guideline development group. The practices were transcribed word by word into the database.

Consolidation
Two initial phases of the first round of Delphi method took a total of 40 min (20 min for individual brainstorm and 20 min for pair-wise discussion).
During the last phase of this round, three members of guideline development group acted as judges and consolidated the items reported by the panellists. One of the members took the lead on grouping identical items and counted the frequency of the repetitions (or votes) of items across the questionnaires. The members agreed upon few naming and wording conventions. As examples, the original wording of the panellists was preserved whenever possible. Judges combined messages in an objective way (ellipsis of words like "lack of ", "good", etc.). When several identical replies were found, the item took a new generic name given by the judges.   The initial consolidated list of items displayed the successful and failure practices in two separate columns and ordered according to the total number of votes. Two initial phases of the first round of Delphi method took a total of 40 min (20 min for individual brainstorm and 20 min for pair-wise discussion).
The consolidation work pursued with the definition of the lessons-learned statements. The items in success practices were directly transformed into lessons-learned statements while those in failure practices were transformed into their positive opposite. For example: "Poor IT skills within the team" become "Good IT skills within the team".
For triangulation purposes, two other members of the guideline development group repeated the entire consolidation work. Finally, the agreement between the judges was checked and the disagreements were discussed until a common view was found (cf. Nevo and Chan, 2007).

Second round of the Delphi survey
The second round of the Delphi survey was completed in three major phases. The first phase aimed to reduce the scope of items to be assessed by the panellists during the second round. In most studies (e.g. Nevo and Chan, 2007;Jaana et al., 2011) it consisted of a narrowing down phase where the least important items were dropped out. Contrarily, this study clustered the lessons learned into functional resembling groups called the empirical guidelines. This clustering analysis established a two-level hierarchy of items (i.e. empirical guidelines and lessons-learned). The clustering phase was firstly conducted by a member of the guidelines development group that was instructed to use a maximum of 10 clusters per domain of DSS development and use, for the purpose of questionnaire simplification. This member proposed the name of the cluster (i.e. the empirical guideline) and further identified the lessons-learned statements included in each cluster. The lesson-learned was grouped in exactly one empirical guideline. The clustering analysis was repeated by 2 other members for the purpose of triangulation and the discrepancies were further discussed until a common consensus was found.
In the second phase, the panellists were asked to rank each empirical guideline in a digital questionnaire according to their perception of its relative importance, into 4 categories: very important, important, meaning-ful, not relevant. They were advised to classify a maximum of 5 empirical guidelines as very important, and 5 other as not relevant. The remaining empirical guidelines should be classified as important or meaningful. The final ranking of the guidelines was then calculated by giving 4 points to each "Very Important" answer, 3 points to "Important", 2 points to "meaningful" and 1 point to each "Not relevant".
The panellists were further asked to assess the lessonslearned statements proposed for the empirical guidelines. They were encouraged to provide suggestions, especially when they express their disagreement on the content.
The second round of the Delphi method involved 18 panellists from 11 countries replying either to on-line questionnaire launched on April 17 th 2012 or submitting the paper copy filled out during the FORSYS workshop in Zvolen on May 10 th 2012 ( Table 2). The percentage of replies from panellists interested in DSS Architecture & Design and Methods & Models of the Model Base remained as in the first round. Yet, the percentage of replies from the knowledge management interested panellists increased about 7% and the percentage of participatory processes-interested panellists decreased about 5%. 67% of the respondents used the digital questionnaire, and the remaining replied with the printed version during the FORSYS workshop conducted approximately a year after the first round. In the second round, 12 new panellists were involved (2 panellists interested on DSS Architecture & Design, 4 in Methods & Models, 6 in Knowledge Management). The total number of countries represented rose to 23. The results from these panellists were considered separately in order to keep the accuracy of the Delphi survey.
In the last phase of the second round the agreement between the panellists were analysed using the Kendall's tau and Spearman's rank correlation. An aggregated ranking was produced and compared to the original rank based on the votes. Then, the guidelines development group completed the description of the empirical guidelines, incorporating the improvements suggested by the panellists.

Results
The 22 pair-wise questionnaires submitted after the pair-wise discussion in the first round provided a total of 354 items, including those repeated over the questionnaires. The panellists reported significantly more The consolidation work of the guidelines development group at the end of the first round grouped the items repeated across the questionnaires. For example, the "role of different actors well def ined" and the "shallow organization (clear responsibilities)" appearing in the list of successful practices on project management were aggregated under "clearly define the responsibilities among the team members". Similar aggregation was done for other domains of DSS development and use. The consolidation work further aggregated items that were addressed in distinct pairwise questionnaires within complementary successful and failure practices. For example, the item "involve stakeholders and users in all phases of DSS develop-ment", included the successful practices "future owner of the DSS involved", "early involvement of decision makers" and "users as project owners" and the failure practices "not involving stakeholders or users representatives", "wrong people involved" and "future owner of DSS not known" that were reported on 3 distinct questionnaires. Some items were moved across domains and grouped to other items considered semantically identical. In particular the items related with costs, team composition and team involvement that were presented across the phases of DSS development were all grouped into the DSS project organization. For example "no budget for training" reported on Documentation & Training was grouped to other identical items, like "small development budget", and "foreseen enough time and budget" into the final statement of lessons-learned "make sure that adequate budget for the entire DSS development phases is granted". This consolidation work lead to the reduction of the total number of practices to 191. After the failure practices were transformed into their positive opposites, the final number of statements, i.e. lessons-learned, was reduced to 102.
After the second round, the "DSS architecture and specification methodologies" (G06) became the most voted with 10 votes as "very important" (and 61 points in the total ranking), while it had rank 6 after the first round (Fig. 2, Fig. 3). The guidelines "adequate team composition size and motivation" (G01), "stakeholders and users involved across the entire DSS development phases" (G26) and "user documentation" (G20) were the most voted in both rounds. The score in the total ranking was 63 points, 57 points and 61 points respectively. When only the very important answers are looked at, the "user-oriented interfaces" (G07) rose to the rank 5, while previously being in the 13 th position. The G09 "DSS development framework" and G30 "Models and Methods within DSS" that were initially in the top five, dropped to the 16 th and 24 th position after the second round. The least votes were given to guidelines concerning the development cycle (G10), price policy (G23) and commercialization structure (G22). They also got most "Not relevant" answers during the 2 nd round. The score in the total ranking was 31 points, 35 points and 39 respectively.
The analysis of the panellists that were involved in Zvolen, but not in Thessaloniki (12 people), shows a similar ranking of the guidelines when considering only the very important answers. The second to sixth f irst ranked guidelines after the second round are within the fifth first ranking positions of these panellists. Only G06 "DSS architecture and specification methodologies" dropped from first to 10 th place. The biggest change in position occurred with G28 "Stakeholders and users motivation towards DSS utilization", which rose from rank 35 to rank 7, as well as with G21 "User training", which rose from rank 32 to rank 9. These huge differences could be explained by the fact that a presentation focussed on stakeholders was presented just before the exercise.
The rank correlations between the 1 st and 2 nd round were 0.69 for Spearman's correlation and 0.50 for Kendal's tau, which shows that the relative importance of the guidelines changed after the second round. The second round emphasised the user's perspective while the first round emphasised the developer's perspective. However, if we compare the ranking between the 4   profiles of the panellists (i.e. academics, developers, government scientists and researchers) ( Table 3), 3 of them ranked the guidelines G06, G20 and G26 as the most important, even the developers. G06, G20 and G26 were considered very important also by the panellists from 3 FORSYS work groups.
Panellists reached moderate consensus also on the importance of "Adequate team composition, size, motivation" (G01), "Modular developments" (G05), and "User-oriented interfaces" (G07) that were regarded as very important by 2 of the profiles. When the results were analysed per panellists' main Work Group of Interest, "User Tests" (G15) were added to "Adequate team composition size and motivation" (G01) as very important guideline by 2 work group of interest. In fact, 8 of the guidelines were among the top 5 voted as very important when the results were analysed by panellists' profile and main FORSYS work group of interest (G01, G02, G05, G06, G07, G15, G20, G26). None of these rankings displayed guideline under the domain of Knowledge Management among the 5 most important.
From the Fig. 4 it can be seen that the second round has enabled the differentiation between the guidelines that had ties in the first round. For instance, a couple of guidelines had 9 votes in the first round, but after the second round their votes varied from 41 to 57. This also indicates that the importance of some items rose after the first round.

Discussion
The results showed that the group communication process relying on the Delphi-survey is a quick way to capture tacit knowledge and to reach consensus on the guidelines identification and their relative importance under different perspectives of large number of experts. Another advantage of this process is that it could be applied for guidelines development in any f ield of research.
When applying this method, the size and composition of the expert panel and the guidelines definition group are of importance. The participants' expertise should cover the range on domains in study, in a proportion representative of the interests of the entire experts community (in this case the participants of the FORSYS project).
The guidelines definition group should have at least 3 members involved in the consolidation phase after the first round. Discussion and informal consensus was needed to tackle few dubious situations, although their effect on the final results was proved to be minor. This was the case of clustering some of the lessons-learned that could have belonged to more than one guideline. For example, "system embedded in the business of the organization" could be assigned to G09 or alternatively to G17, which is related to FMDSS maintenance. Moreover, some guidelines could be in more than one domain. For instance, "Involvement of users and stakeholders on project management" can belong to domain DSS Project Organization or to Stakeholders Engagement. Another issue that emerged during the Delphi survey was that some lessons-learned could be considered themselves as an empirical guidelines, or could be grouped into a more general guideline. For example, the "development tests", "performance tests" and "user tests" were initially grouped into a single guideline related with DSS tests. However, after discussion, the guidelines definition group decided to make separate guidelines in order to raise the importance of the user tests perceived by the panellists.
One reason for the change on the relative importance of the guidelines in the second round may lie in the way the questionnaires were formulated. The questions of "what went well" and "what went bad" in the first round may have led the experts to thinking in terms of the development process in the purest sense, while the compiled list of empirical guidelines in the second round could have reminded the experts about the more general topics of DSS development. It is evident that the second round has increased the value of the empirical guidelines by encouraging the panellists to give more thought to the items.
The outcome of the process may provide valuable insights for future FMDSS developments. The guidelines may contribute to more efficient DSS development processes through relevant recommendations particularly in the domains of DSS project organization and DSS development.
In respect to the domain of Models and Methods, the lessons-learned do not intent to provide much information concerning which specific methods or models should be included in the DSS. On the other hand, this may indicate that the possibility to select the type of growth model is not a main concern, or since growth and yield models are already implemented and represent the core of forest DSS, they do not need to be mentioned.
Likewise, the experts think it is important to have some knowledge management tools within a DSS, but only 10 questionnaires provided successful or failure practices in respect to the Knowledge Management techniques, which may evidence that there is low experience of the FORSYS community about this domain. No information concerning the specific tools was detected. For instance, having a GIS and a database is probably seen as prerequisites for using a FMDSS, and as such self-evident, while other KM methods seem to be too poorly known among the forestry experts to give any proper guidelines as to their use. Furthermore, the panellists clearly perceived the importance of the stakeholder engagement. They gave some recommendations of the composition of stakeholders and users group but without reporting specific participatory methods applied to enhance their involvement.
The failure practices reported during the first round were also an important outcome of the process; they generally reflect some aspects affecting the success of the developers' project, but may still be neglected or need significant improvement. In particular, the pa-    nellists expressed the need to improve "DSS project organization in respect to the team composition, size and motivation" (G01) as well as "Project planning and budgeting, foreseeing the continuity after DSS development" (G04). They reported def iciencies in the dissemination and commercialization structures (G22 to G24), which fail to bring the DSS close to attention of the potential users. These growing concerns of the developers may be a consequence of evolving, in some cases, from prototyping done by a small team under the framework of research and academic projects, to fully integrated DSSs developed by larger teams often for commercial purposes. Fürst et al. (2010) reported that altough computerized-tools are clearly prefered to support planning and decision processes, the use of subjectively "home-made" combinations of fragmented solutions (spreadsheets for calculation, mailing for comunication, GIS for visualization and spatial analysis) still prevails. This suggests that the challenge is to develop more integrated and robust solutions.
The panellists further reported inefficiencies on the DSS development process, related with long DSS development cycles and "black-box" development frameworks that may be due to the use on inappropriate specif ication methodologies (G06), lack of code documentation (G12), or lack of standards and code reutilization (G09).
The failure practices further provided evidences about other aspects of the DSS development that may impact its usability to the final users. They highlighted the need to focus on the users, meaning that the DSS should address the users' requirements, favouring simple interfaces easily apprehended by the users rather than research-oriented interfaces (G07). Previous work on the users' requirements in respect to tools to support forest management (Fürst et al., 2010) also suggested that self-explanatory user interfaces are a precondition for broad acceptance and use. They further emphasized the importance of broad and instant accessibility to users, i.e. provision of an on-line service or on-line support. In our research, the aspects related with web applications were grouped in the guideline "DSS development framework" (G09) that had 1 vote as "very important" (corresponding to the 13 th position on the ranking) but 11 votes as "important". Fürst et al. (2010) stated that three most important features of DSS from the user perspective are the possibility to integrate iteratively experience from case studies, from regional experts as well as future scientific results into a knowledge management tool (i.e. learning system). The empirical guideline "KM tools within DSS" (G34) expressed this concern but this aspect was considered very important only by 5 of the panellists.
In order to improve the FMDSS usuability, the panellists suggested that the stakeholders and users should be involved in all DSS development phases. Particular attention should be given to users' training and the DSS user tests, preferably with real data (G21, G15). The development team should provide user documentation (G20) and user support, forward periodic DSS updates and also commit to a affordable price policy (G23).

Conclusions
This article presented a group communication process for the definition of guidelines for Forest Management Decision Support Systems development and use.
The process is replicable and further assures the consistency and validity of the arising guidelines. It relies on a Delphi-survey approach to capture past experiences. It proved to be adequate to capture tacit knowledge from a group of experts. Consequently, it may be used complementary to other methods used to synthesize past experiences, such as case studies, reviews and other types of surveys.
This research was the f irst known initiative to establish guidelines in the field of natural resource management. It involved a total of 30 experts from 23 countries in the FORSYS EU-COST project. It led to the identification of 37 empirical guidelines that group 102 lessons-learned. The guidelines cover several domains, including DSS Project Organisation, DSS Development, Models and Methods, Knowledge Management Techniques, and Stakeholders Engagement, their Role, and Adoption on Real Life Situations. They address both the developers and the user's perspectives.
This research showed that the empirical guidelines driven from the past experiences may be valuable for future FMDSS developments. In future, the development of FMDSS should be seen as a communicative process between the multidisciplinary development team and stakeholders and users, which should be involved throughout the entire FMDSS development. DSS architecture and specification methodologies are still likely to play one of the key roles in successful functioning of the DSS, but the developers should put more efforts in initial problem definition and problem structuring, and they should favour methodologies that would be able to adapt to future users' requirements and would also meet changing policy and market requirements. Such adaptive communicative process should finally result also in adequate user documentation and more user-oriented interfaces and thus hopefully satisfy the end-users.
The empirical guidelines definition process will proceed with the establishement of a knowledge repository built on a wiki web-based platform (http:// fp0804.emu.ee/wiki/). Future work will focus on the content maintenance of this web page. The guidelines will be freely available and the forestry community world-wide may consult them and therefore improve their descriptions. New lessons-learned and/or guidelines may be suggested and added to the wiki after discussion among the guidelines definition group.
The community may also be asked to assess the quality and relevance of the empirical guidelines by rating several evaluation criteria displayed in a on-line rating tool embedded on the wiki. This future tool may be inspired on other appraisal tools in use to assess practice guidelines (e.g. the AGREE II tool, 2009). Appendix 1. List of empirical guidelines and their underlying lessons-learned. Includes the number of votes in each lessonlearned at the end of each Delphi round. The votes of 1 st round represent the number of items reported on the pair-wise questionaires that were then included it the empirical guideline. The votes of the 2 nd round were calculated by giving 4 points to each "Very Important" answer, 3 points to "Important", 2 points to "meaningful" and 1 point to each "Not relevant" Adequate project planning, -Elaborate an overall plan for the entire DSS development 9 49 budgeting, implementation, process: and support a) Account for testing, documentation, training b) Foresee the continuity of the DSS after the end of the project c) Plan for support services -Revise the DSS development plan frequently to account for delays -Ensure that adequate budget for the entire DSS development phases is available

G05
Modular development -Use plug-in architecture, supported by modules that: 12 56 a) Can be easily integrated into more complex systems b) Can be combined into a flexible DSS appropriate for the intended application c) Enhance the DSS re-utilization in new problems and cases -Foresee DSS scalability to larger problem instances and new problem types G06 DSS architecture and -Use brainstorming for thorough problem description specification methodologies before starting the design and programming 17 61 -Favour the DSS design and architecture methodology which: Appendix 1 (cont.). List of empirical guidelines and their underlying lessons-learned. Includes the number of votes in each lesson-learned at the end of each Delphi round. The votes of 1 st round represent the number of items reported on the pair-wise questionaires that were then included it the empirical guideline. The votes of the 2 nd round were calculated by giving 4 points to each "Very Important" answer, 3 points to "Important", 2 points to "meaningful" and 1 point to each "Not relevant" Id. Empirical guidelines Lesson-learned 1 st round 2 nd round a) Enhances the involvement of stakeholders and users b) Is based on process identification and in the problem description c) Is adequate for both current and future development d) Can guide the entire DSS development process -Provide detailed but efficient specifications that: a) Rely on graphical and interactive techniques (e.g. use cases, E-R models) b) Meet policy and market requirements G07 User-oriented interfaces -Use graphical user interfaces for reporting that are simple, 10 59 user-oriented; with a user-friendly design (distinct from research-oriented interfaces) -Visualize results in geographical information systems (GIS) 5 49  (2013) 22(2), 320-339 Appendix 1 (cont.). List of empirical guidelines and their underlying lessons-learned. Includes the number of votes in each lesson-learned at the end of each Delphi round. The votes of 1 st round represent the number of items reported on the pair-wise questionaires that were then included it the empirical guideline. The votes of the 2 nd round were calculated by giving 4 points to each "Very Important" answer, 3 points to "Important", 2 points to "meaningful" and 1 point to each "Not relevant"

Id.
Empirical guidelines Lesson-learned 1 st round 2 nd round -Have an adequate data set at disposal for performance tests: a) First use a simple case where the results are known b) Then repeat tests with at least one complete data set c) Include both real and historical data, preferably provided by the stakeholders d) Include exceptional cases in the tests -Ensure having experts for performing the tests and/or evaluating the results

G15
User Tests -Let the users conduct tests to assure the compliance 10 57 with their requirements: a) Involve multiple independent users with different profiles b) Conduct the test using the stakeholders' data c) Avoid user tests done by researchers involved on the project d) Plan mechanisms to motivate users to conduct the tests

G16
Team for maintenance -Establish a maintenance and support team with members 9 54 and support devoted to improvement of the tool; representatives from the DSS development team; and the contact person clearly indicated G17 Tools for maintenance -Use specific tools for reporting problems and supporting 6 50 and support requests -Provide on-line support G18 DSS releases and updates -Provide periodic releases based on continuous 5 46 requirements collection and system updates G19 Support Users community -Promote networking and foster a users' community 7 44 -Set up a web page devoted to the DSS and update it with the information for the users' community -Promote "User days" bringing together the users community and the DSS developers

G20
User documentation -Provide user's manual that is short, searchable, 24 61 and preferably on-line; Built with examples and case descriptions; Uses language easily understood by the users; Suitable for different user profiles; Developed with the help of users (users to users) -Add help buttons on the dialog boxes & wizards -Provide a demo version with sample data -Ensure that the DSS is published in scientific articles

G21
Users training -Organize interactive training with follow-up 5 48 -Support training on-the-job -Enable self-learning

DSS Development-Dissemination & Commercialization G22
Commercialization structure -Establish a commercialization network (national and 13 39 international partners and customers) -Use simple and appealing DSS interfaces to motivate interested users Appendix 1 (cont.). List of empirical guidelines and their underlying lessons-learned. Includes the number of votes in each lesson-learned at the end of each Delphi round. The votes of 1 st round represent the number of items reported on the pair-wise questionaires that were then included it the empirical guideline. The votes of the 2 nd round were calculated by giving 4 points to each "Very Important" answer, 3 points to "Important", 2 points to "meaningful" and 1 point to each "Not relevant" M&M well documented -Use M&M well described in the scientific literature, 2 49 and scientifically sound avoid using "black-box" models G32 Select the best fitting M&M -Combine several interoperable M&M within the same DSS 16 58 to the problem and the user's -Favour the use of robust M&M that are adequate for needs several problems and cases -Select M&M appropriate for a specific problem -If the DSS relies on a separate commercial component (e.g. a linear programming solver), try to make the DSS adaptable to multiple vendor solutions -Incorporate M&M that handle multi-criteria decisions -Incorporate M&M that handle spatial data and spatial constraints -Use IT infrastructure that enhances the M&M performance even for complex problems -Take into consideration the quality of the information available for the M&M G33 M&M easy to understand -Produce output of the M&M that is easily interpretable 1 42 and interpret by the users