Improving Productivity by the Automation of Checking of 3D Parametric Modelling

Improving Productivity by the Automation of Checking of 3D Parametric Modelling

Dene R. Warren Noha Saleeb

Department of Design Engineering & Mathematics, Faculty of Science and Technology, Middlesex University, Hendon, London NW4 4BT, UK

Corresponding Author Email: 
n.saleeb@mdx.ac.uk
Page: 
441-450
|
DOI: 
https://doi.org/10.18280/ijsse.100402
Received: 
15 April 2020
|
Accepted: 
3 August 2020
|
Published: 
31 August 2020
| Citation

OPEN ACCESS

Abstract: 

The development of object-based 3D modelling based on Industry Foundation Class (IFC) has given rise to the possibility of computer-based interpretable models and the automation of checking based on rules. Past studies suggest that professionals within the industry have low confidence in the quality of design data, and consequently, there is an over-reliance on manual checking. This study aims to examine the application of automatic checking in industry and opportunities to improve productivity by automation. It identifies a means to improve the quality of data and available checking software, albeit with poor interoperability. The study indicates a quality system that can be applied to the generation of checking rules and a method to improve productivity of automatic checking by using the principles of Dataflow.

Keywords: 

3D-model, automatic, BIM, checking, parametric, productivity, quality, rule-based

1. Introduction

The architectural, engineering and construction (AEC) Industry has recognised that savings can be achieved in whole-life costs of building assets by improving the quality of information available. Here quality is defined as conformance to standards and the client's requirements [1]. Thinking is shifting from 3D visualisation towards the speed of information flow, and accuracy [2]. As speed and complexity increase, the industry is seeking increased efficiency by the automation of checking. Automated checking is defined here as algorithmic software that does not modify a building design but instead assesses it by the configuration of objects, their relations or attributes defined by rules [3].

The quality of automatic checking will depend upon the quality of the rules used, and the quality of manual checking will depend upon the experience of the checker and the amount of time spent. Automatic, rule-based checking can bring consistency, quality and speed, whilst manual checking ensures an expert review is applied in a domain-specific context; clearly, a balance is needed. Of course, one way to improve quality is not to make errors in the first place; right first time etc. To this end, it is common practice for a design team to establish a library of objects complete with correct parameters. In practice, however, objects may need an amendment to suit individual circumstances, and the control of an object library becomes fundamental to the quality of the final product. Once a 3D model has been processed by various domain disciplines and the supply chain, a check will be required.

Past studies suggest that professionals within the industry have low confidence in the quality of design data measured by the percentage of re-work [4]. This study investigates the current use of automatic checking methods and reasons why there is an over-reliance on manual checking. It identifies a means to improve quality by the application of a quality framework that can be applied to data within 3D models and is supported by software, albeit with poor interoperability. The study indicates a quality system that can be applied to the generation of checking rules. The study also indicates a means to improve productivity of automatic checking by using the principles of Dataflow, (See section 4.0).

2. Literature Review

The research into methods of generating rules for checking has been broadly divided between two schools of thought: manual logic-based interpretation and language-based interpretation into computable form. The former is to apply individual manually written Boolean logic tests to represent the constraints and test for compliance [3, 5]. The latter is Natural Language-Based Interpretation [6, 7] or Natural Language Processing (NLP) technique [8, 9], using semantic web methods to generate rules. NLP enables a computer to interpret existing standards and codes and automatically derive logic tests for compliance. Regulatory codes can be converted into logic rules in a fraction of the time it takes to convert manually. Since the text is so closely associated with the generated rule logic, it can be considered as a complete quality system. NLP techniques could revolutionise rule-based checking software but are yet to become commercially available. While both processes can generate rules that give automatic alerts due to non-compliance, they remain dependent on manual intervention to report and correct the design. The ultimate aim is to automate the system entirely [10].

2.1 Quality framework for data

Quality control can be applied to data within 3D models by following the guidance within PAS1192 (Specification for information management for the capital/delivery phase of construction projects using building information modelling 2013) [11], BS EN ISO 19650 (Organization and digitization of information about buildings and civil engineering works, including building information modelling) [12], BS EN ISO 9001 (Quality management systems: requirements) [13], and BS EN ISO 29481 (Building information models - Information delivery manual - Part 1: Methodology and format) [14]. Here quality control is an activity intended to demonstrate compliance with standards and requirements [15].

The quality framework [16] is an assemblage of the above. Semi-automatic checking is provided by discrete executable programmes such as Navisworks, SMC, CobieQCReporter and ifcDoc. Whilst each programme is efficient within itself, manual intervention is necessary to effect data transfer between programmes so introducing the potential for error and a reduction in productivity.

2.2 Quality system for authoring rules

The quality of automatic checking depends upon the quality of the checking rules used, and so a quality system for generating checking rules is fundamental to the quality of the final product. Whilst there is no formal guidance on authoring rules, guidance is provided by Hjelseth [17], Bell et al. [18] and supported by industry specialists such as Bond Bryan [19].

Building on the work of Hjelseth [17], in 2009, Bell et al. [18] suggested a six-stage method of development of computable rules. The stages include 1 Scope definition, 2 Define the structure of the code, 3 Create computable rules, 4 Verify, 5 Transfer to logic notation, 6 Test, and 7 Certification. Based on the principles of knowledge-based engineering, it is noted that each stage is interactive, and iteration is essential for effective application.

2.3 Software

Manual checking ensures an expert review is used in a domain-specific context, avoiding false positives and multiple negatives. Research has shown, however, that the cognitive challenges of drawing-based checking are such that even experienced professionals are inconsistent [20] and can detect only a small fraction of errors present [21]. Efforts to improve productivity and accuracy will include greater use of automatic checking software.

Computer-based automatic checking programmes promise consistent, rapid verification of data. They use pre-set rules and applied logic, the quality of which is fundamental to the quality of the finished product. Software such-as Solibri™ Model Checker (SMC) use 50 or so 'out of the box' rules that can be amended or augmented to suit individual needs, but this can be problematic. Expert knowledge is often required to interpret the meaning of the rules in context with domain-specific assumptions, general knowledge and the combinatory effect of related rules [22]. While automatic checking is efficient, each step is served by specialised checking software applied semi-automatically. This introduces the opportunity for error and reduced productivity. An opportunity to automate building information management (BIM), improving quality and productivity is given by Dataflow programming [23].

2.4 Automation of the quality framework

Dataflow programming allows data to flow between discrete operations (executable programmes); thus, implementing dataflow principles of taking the data to the operation rather than the operation to the data. Implementation can either be data-availability-driven (event-driven) or data-request-driven (demand-driven) [23]. Improved efficiency can be depicted as the current flowing in a circuit between electrical components, so data flows between operations in series or parallel. Dataflow programming environments such as LabVIEW [24] and Microsoft Azure [25] are widely used in various industries. Dataflow environments intended for the AEC industry are tailored for model authorship platforms, and automatic programme interface (API)’s such as Geometry Gym [26] and Grasshopper [27].

2.5 BIMSie

BIMSie is an open-source framework proposed by Aerle [28] that allows creating an ‘envelope' for formal, graphical and flexible representations of dataflow processes that can be shared and re-used across projects based on the BIMSie API. The BIM Service interface exchange (BIMSie) [29] is designed to connect BIM web services (such as BIMQL, and mvdXML checker) software authorship platforms, and Common Data Environment (CDE)'s in an automated event-driven (data-driven) fashion. The goals of the project are to automate interaction between online BIM services, giving the possibility to innovate with BIM workflow in the ‘cloud’.

2.6 Developing dataflow

Automatic checking has traditionally been focused on clash detection and data checking with SMC, and other software platforms are now becoming available for output checking such as ifcDoc and COBie QC Reporter. Dataflow programming has the potential to link checking programs and CDE into an integrated information management system.

Figure 1. Manual input (shown coloured 80%), automatic (shown clear 20%)

Figure 2. Manual input (shown coloured 20%), automatic (shown clear 80%)

Current practice is fragmented with each of the quality framework steps separately defined and underpinned by the approval process. Checking is manually instigated, and results are applied manually. The current balance between manual and automatic checking is shown diagrammatically in Figure 1.

Checking of the technical content is a cycle of output checks, applied by standard methods and procedures. The cycle is applied manually with small parts of the system supported by the common data environment automatically. Development of a rigorous quality system applied to check rules will offer more confidence in the results, and better-checking rules will enable more automation.

The combination of dataflow programming and better rules for automated checking will enable reduced effort for better quality. With the move towards automatic cloud-based dataflows with the likes of BIMie and better checking supported by technology such as NLP techniques [8, 9], the effort necessary to achieve a quality product will be reduced. Figure 2 shows the advantage of reduced effort by reversing the current balance of manual to automatic checking using Dataflow.

2.7 Literature review summary

While automatic checking is efficient, each step is served by specialised checking software with poor interoperability. Expert knowledge is often required to interpret the meaning of the rules in context with domain-specific assumptions which introduces the opportunity for error and reduced productivity. The literature review has established a means to improve productivity in the short term by the application of a quality system that can be applied to checking rule authorship. The study also identifies how the principles of Natural Language Processing (NLP) and Dataflow have the potential to improve productivity in the long term.

3. Consultation with Industry

Having established the existence of a quality framework, and opportunities to automate checking, it is necessary to gain some understanding of its application by consultation with the AEC industry. As part of the author’s research work on this topic, a survey was carried out using a combination of interviews and a questionnaire to gather data.

3.1 Data collection methods

In an effort to introduce impartiality, the method of triangulation [30] and the checklist of quality [31] were used as the basis of the survey design. Three methods of data collection were chosen, the questionnaire, two expert interviews and a focus group. The questionnaire maximised the potential population of a specialised discipline and provided a quantitative method of data collection. Expert interviews and a focus group were chosen to validate the data collected via the questionnaire to ensure reliability and integrity. The questionnaire was released before the expert interviews to enable discussion and in anticipation of a slow response. It used the eSurv platform as recommended by Farmer et al. [32] between November 9 to December 9 2018, with an assurance of anonymity and compliance with GDPR.

Two expert interviews were arranged on a one to one basis, followed by the focus group. Use of a broad range of experience in the AEC associated with automatic rule checking was achieved by consultations with experts of strategy (first interview AEC3), technical (second interview Solibri™) and operational (focus group with Mott MacDonald Ltd).

3.2 Sampling data for the questionnaire

Stratified/cluster sampling of the population was chosen for the questionnaire to include three groups:

  • An IFC's special interest group within LinkedIn social media consisting of 4859 members.
  • Employees within Mott MacDonald Ltd (MML). MML is an international engineering company of an estimated 3500 employees involved in AEC across the world.
  • Students at Middlesex University (MDx), currently involved with the MSc in BIM and consisting of 194 persons. These students are full-time practitioners within the AEC industry.

The total population size approached to take part in this survey is estimated at 8553. According to the Raosoft sample size calculator [33], based on a normal distribution, the recommended sample size is 68, for a margin of error of 10% and a confidence interval of 90%.

Respondents total number was 92 with a confidence level of 95% and a margin of error of 10%. 40 MML, five from LinkedIn and 47 from MDx.

3.3 Data analysis

Univariate analysis using descriptive statistics [34] was used to reflect the quantitative nature of the questionnaire. It included a brief exercise of correlation by linear regression of the questions that adopted the Likert scale [35]. A six-phase thematic analysis [36] of the data was used to reflect the qualitative nature of the data gathered from interviews. It endeavoured to trace patterns across qualitative data, termed thematic discourse, to formulate the opinions of the group or individual, as shown in Figure 4.

3.4 Questionnaire design

The questionnaire was based on the hypotheses that there is a need for automation, development and understanding of checking data. The mapping between these, the survey questions and the questionnaire are shown in Figure 3.

Consistent data collection was achieved by careful wording of the questions to be objective and relevant to the study. In general, the questions were cognitive [37] or informative. [38], and worded in a predominantly closed style [38].

For consistency; and ease of participation, questions were designed as multiple-choice, checklists, and where possible, used the Likert scale [35]. Some questions contained the option for an open response (i.e. other).

3.5 Interviews and focus group design

Interviews were designed to allow inductive reasoning through the qualitative analysis of data collected from experts working within the AEC industry who have experience in BIM. The aim was to gain a better understanding of the quality of rule-based checking from the perspective of the interviewee/s. So, a semi-structured (guided) interview style was adopted where a pre-prepared set of questions were followed, but the order could be changed, questions missed or added. Both individual interviews and a focus group were used to validate the reliability of the questionnaire data, thus adopting a method of triangulation, with the added advantage of a ‘small group’ synergy of ideas [39].

Participants were chosen within the field of expertise of BIM but with different roles:

  • The first interview was with an expert in automatic rule generation using natural language programming from AEC3 UK Ltd.
  • The second interview was with an expert from Solibri™. One of the founding members of the company is regarded as an expert in logic-based rule authorship and is mostly responsible for the Solibri™ add-on module for COBie compliance checking.
  • A focus group attended by employees of Mott MacDonald Ltd involved the operational application of BIM.

Pre-defined questions were sent to participants before the interviews, and the duration of interviews was between 60 to 90 minutes. With permission, the discussions were recorded, transcribed, and verified by the participants.

The format of the sessions included an introduction, a general discussion of relevant experience before going through the pre-defined questions and a summary. Manual transcription of the interview recording is a useful way to draw-out themes and statements which can be highlighted and mapped to questions and answers. The structure of the questions and the thematic analysis is shown in Figure 4.

Figure 3. Research questions mapped to the questionnaire

Figure 4. Thematic analysis

4. Results

4.1 Results of the questionnaire

4.1.1 Existing quality of design

Past studies have suggested that AEC professionals have low confidence in the quality of design data [4], and the results of this study continue to support that view. The first questions asked respondents to judge the current standard of design information and if it was changing. The result shows that most (42%) thought the standard of building information is poor, but with an almost equal number saying it is acceptable (40%); a majority thought that accuracy was improving. The results are separated by experience, sector, location, and role. They indicate that current standards are judged as poor by the largest category of respondents who have 16 to 25 years’ experience. See Figure 5.

4.1.2 The opinion of the cause of poor standard and solution

Question 4 and 5 requested an opinion as to the cause of poor standards and how to improve this. Results for Qu4 were evenly spread. Lack of coordination, and a lack of time for checking, were given as the main problem areas. Poor scope definition, poor modelling skills, and ineffective quality control were also seen as contributory factors followed by the lack of IFC knowledge, Figure 6 (a). Results for Qu5 indicates that better training and software support is required. See Figure 6 (b).

Figure 5. Opinion of existing standard of design (Qu 1&2)

Figure 6. Opinion of the respondents (Qu 4&5)

4.1.3 Balance between manual and automatic checking

Question 6 asked respondents to estimate the current balance between manual and automatic checking, and question 7 asked if they could see this changing in the future. The majority of respondents indicated a balance of 80% manual to 20% automatic. Most respondents expect to make more use of automatic checking in the future. The results are also separated by experience, sector, location, and role, each indicating a majority have a checking balance of 80/20 (manual/automatic). See Figure 7.

Figure 7. Balance of manual to automatic checking (Qu 6)

4.1.4 Confidence in existing rules

Question 8 and 9 of the survey asked respondents to indicate a measure of confidence in existing checking rules. Figure 8 (a) shows that most respondents indicated that the rules could be much improved, implying a lack of confidence in the existing provision. As for authorship of checking rules, only a minority said they used out-of-the-box rules, with the majority either amending existing or writing their own rules. See Figure 8 (b).

4.1.5 Quality system (Qu 10&11)

The survey asked respondents if they use a system of validation and approval for amended or newly written rules in checking software? The majority of respondents indicated that they either had a voluntary validation system or no validation at all. Only a minority had formally trained competent staff for validation and approval of rules, and the majority dealing with this had gained competence by experience alone. The results were also separated by experience, sector, location, and role, and this indicated that the largest group with no formal validation is the group who have 16-25 years’ experience of design in the commercial sector and in the UK. See Figure 9 (a) and 9 (b).

4.1.6 Proposal for improvement (Qu 12)

The survey asked respondents to identify the key areas for action that could improve quality. Feedback was spread across all areas with no overall lead. 23% indicated better training, 20% better quality control, 14% more resource, 11% more automatic checking and 11% better modelling skill. See Figure 10.

Figure 8. Indication of confidence & authorship (Qu 8&9)

Figure 9. Do you have a quality system and competent people? (Qu 10&11)

Figure 10. Proposed key areas for action

4.1.7 Software (Qu 13)

The survey asked for the identity of the checking software used, and the most popular was 38% Navisworks Manage for geometry checking and 27% Solibri™ for geometry and data. See Figure 11.

Figure 11. Software

4.1.8 Details of respondents (Qu 14, 15 16 & 17)

The closing questions of the survey asked details of respondents, as shown in Figure 12 (a) and (b).

4.1.9 Correlation

Correlation of results showed there was a trend towards a poor and acceptable quality of design information for those who predominantly adopt manual checking. The trend is also towards a poor and acceptable quality of design information for those who have a voluntary validation or no validation of rule authorship in place. Figure 13.

4.1.10 Limitations

Total returns of 92 indicate a confidence level of 90%. Due to the limited sample size, the margin of error is approximately 9% using the Raosoft sample size calculator [33].

(a) Location and experience

(b) Sector and role

Figure 12. Respondent details (Qu 14, 15, 16 & 17)

Figure 13. Correlation

4.2 Focus Group results (Mott MacDonald)

The Focus Group discussed checking throughout the design process to generate a good quality 3D model. Currently, this involves considerable manual effort from staff at all levels and all process stages. The 3D model is coordinated by designers using Bentley Navigator and data managed with CadQA. However, because of interoperability issues, they are using interlinked spreadsheets and checklists using PowerBI to test data. The workflow system is within ProjectWise and controls verification validation and approval. Competency is assessed internally, and access permissions are regularly reviewed and reported. The focus group agreed that the quality of design building information could be improved and that the potential for rule-based checking is not fully realised. They indicated a lack of software support for infrastructure orientated design and a lack of understanding of IFC generally. IFC currently poorly supports infrastructure design with platforms such as BentlyBE, and Revit, which is still operating in IFC2x3. IFC5 which will cater to infrastructure is currently under development. However, in the meantime, mapping is required as an IFC Proxy such as buildingSMART MVD for LandXML v1.2 or IFC (Stub) - IFC4x1 Alignment Extension. The Focus group agreed that automated checking could have a beneficial effect. They estimated that the balance between manual and automatic checking is about 80/20 but with the aspiration of more automation.

4.3 Expert interview results (AEC3 and Solibri)

The expert interviewees agreed that the quality of building information could be improved and that the potential for rule-based checking is not fully realised. Expert interview with Solibri highlighted the lack of understanding of the IFC schema in the industry and a lack of attention to detail. Solibri indicated a perception of lack of understanding of rules, rule authoring and crucially a lack of a quality framework.

Expert interview with AEC3 reinforced the lack of current means and the difficulty of rule authoring. AEC3 agreed that automatic checking is necessary and that the need for development is embodied in NLP. AEC3 did not believe that the amount of manual checking would be reduced, but the balance may swing towards automatic.

AEC3 emphasised the potential of NLP to generate rulesets. These are so closely linked to the regulatory code to represent a quality system within itself.

5. Discussion

This review of academic research, technology and standards shows that although there is guidance available for a quality framework, there are difficulties associated with the application of rule checking. Results of the questionnaire, supported by expert opinion, show that the main difficulties are the lack of training and the complexity of rule authorship. For examples, see Parametric Modelling in Construction: Investigating the Quality of Rule-Based Checking [16].

5.1 Authorship of checking rules

The complexity of rule authorship is often underestimated, and the application of a quality system by design professionals inconsistent [4, 16]. Expert knowledge is often required to interpret the meaning of the rules in context with domain-specific assumptions, and manually write Boolean logic tests to represent the constraints and test for compliance [3, 5]. NLP has the potential to automate rule production, reliably and in a fraction of the time needed for a manual rule interpretation. The technology will enable new and revised codes to be quickly incorporated into automatic checking programmes. The quality of the interpretation will also enable the extension of automatic rule checking into areas not currently covered commercially, e.g. regulatory code compliance, 4D, safety and more. While there are distinct advantages of rigorous code compliance checking through either SMC or dataflow programming its practical application is as yet unproven.

5.2 Training

There appears to be little formal training available for personnel involved in managing building information, in the checking of data and particularly in generating rules for automatic checking. Feedback from the industry indicates that little thought has been given to validating and approving new or amended checking rules. This training would have to encompass the entirety of building information management but with a focus on managing automatic checking software and validation. Short industry training courses are available, but the application is specific, its general effectiveness unknown, and subsequent certification is missing. Certification programmes for rule authoring, the management of rule authoring, validation and approval are not yet available. Market research of the potential may prove to be of commercial interest.

6. Conclusion

This study has considered automatic checking, how it is currently applied in industry and opportunities to automate. Review of the research literature, software and procedures have shown that rule authoring is a lengthy process, can be complicated, and is error-prone. The research has shown that there are examples of quality frameworks that can be applied to rule authoring [16]. Findings support the hypothesis that there is a need for checking automation, a means to achieve automation and a need for a better understanding of generating rules for automatic checking.

The questionnaire, expert interviews and the focus group confirm a poor standard of data in design. The balance is currently overwhelmingly in favour of manual checking but with the perceived need for more automation. Correlation of results shows a trend towards the poor and acceptable quality of design information. The indication is that out-of-the-box rules are used without a formal validation, and rules are being authored without formal training or assessment of capability. Feedback implied a perceived lack of understanding of rule authoring and crucially a lack of an applied quality framework, and the expert interviews confirmed this. Most respondents envisage improvements in accuracy through future utilisation of automatic checking.

The focus group indicated a lack of good software support both for IFC and interoperability, and this was reflected in the questionnaire. Both emphasised the lack of time and the need for automation to speed up turn-around, highlighting the lack of software support for accurate IFC extraction. Expert interview with Solibri™ also sighted the lack of understanding of the IFC schema in the industry and the lack of attention to detail. AEC3 reinforced the lack of current means of rule authoring. AEC3 agreed that automatic checking is necessary and that the need for development is embodied in NLP.

Over-reliance on manual checking and a lack of formal training in generating checking rulesets could be the cause of the poor standard of information found in 3D models. Open standard automatic cloud-based dataflows supported by the BIMie project [28] and better checking rules supported by NLP techniques [8, 9] have the potential to increase the productivity of design checking, verification and approval

Whilst there are obvious advantages of rigorous code compliance, through SMC, NLP and Dataflow, checking the combinatory effects of multiple rulesets and their linkage is yet unknown. Research into types of interpretation and contextual inputs is required. Market research of the potential training may also prove to be of commercial interest.

Acknowledgements

Thanks to Dr Noha Saleeb, for guidance, AEC3 Ltd, Solibri™, and Mott MacDonald Ltd, for attending interviews and all those in the AEC industry who took the time to complete the questionnaire.

  References

[1] Crosby, P.B. (1984). Quality Without Tears: The Art of Hassle-Free Management. 1st (reprint). McGraw-Hill.

[2] Hardin, B., McCool, D. (2015). BIM and Construction Management: Proven Tools, Methods, and Workflows. 2nd Kindle Edition. Wiley. 

[3] Eastman, C., Lee, J., Jeong, Y., Lee, J. (2009). Automatic rule-based checking of building designs. Automation in Construction, 18(8): 1011-1033. https://doi.org/10.1016/j.autcon.2009.07.002

[4] Suermann, P.C., Issa, R.R.A. (2007). Evaluating industry perceptions of building information modeling (BIM) impact on construction. Electronic Journal of Information Technology in Construction, 14: 574-594.

[5] Lee, Y.C., Eastman, C.M., Lee, J.K. (2015). Validations for ensuring the interoperability of data exchange of a building information model. Automation in Construction, 58: 176-195. https://doi.org/10.1016/j.autcon.2015.07.010

[6] Dikbas, A., Ergen, E., Giritli, H. (2009). Collection of Papers: Managing IT in Construction/Managing Construction for Tomorrow. CRC Press.

[7] Park, S., Lee, H., Lee, S.I., Shin, J., Lee, J.K. (2015). Rule checking method-centered approach to represent building permit requirements. The 32nd ISARC, Oulu, Finland, pp. 1-8. https://doi.org/10.22260/ISARC2015/0049

[8] Salama, D.M., El-Gohary, N.M. (2011). Semantic modeling for automated compliance checking. International Workshop on Computing in Civil Engineering 2011, Miami, USA. https://doi.org/10.1061/41182(416)79

[9] Beach, T.H., Kasim, T., Li, H., Nisbet, N., Rezgui, Y. (2013). Towards automated compliance checking in the construction industry. Database and Expert Systems Applications, 8055: 366-380. https://doi.org/10.1007/978-3-642-40285-2_32

[10] Solihin, W., Eastman, C. (2015). Classification of rules for automated BIM rule checking development. Automation in Construction, 53: 69-82. https://doi.org/10.1016/j.autcon.2015.03.003

[11] Uchiha, S. (2013). Specification for information management for the capital/delivery phase of construction projects using building information modelling.

[12] BSI. ISO 19650-1 Organization and digitization of information about buildings and civil engineering works, including building information modelling (BIM) — Information management using building information modelling — Part 1: Concepts and principles 2018.

[13] BSI. BS EN ISO 9001:2015 Quality management systems: requirements. London, UK: BSI Group; 2015.

[14] BS EN ISO 29481-1-2017--[2018-08-28--03-02-27 PM].pdf n.d.

[15] PMI. A Guide to the Project Management Body of Knowledge (PMBOK® Guide). Kindle Edition. Project Management Institute, Inc.; 2016.

[16] Warren, D. (2019). Parametric modelling in construction: Investigating the quality of rule-based checking. WIT Transactions on The Built Environment, 192: 57-68. https://doi.org/10.2495/BIM190061

[17] Hjelseth, E. (2009). Foundation for development of computable rules. In: Dikbas, A., Ergen, E., Giritli, H. Managing IT in Construction/Managing Construction for Tomorrow. CRC Press.

[18] Bell, H., Bjørkhaug, L., Hjelseth, E. (2009). Standardized Computable Rules. Standards Norway, Oslo, Norway.

[19] Bond Bryan. Testing openBIM data exchange for client specific data sets | Bond Bryan BIM. BIM Blog 2018. https://bimblog.bondbryan.co.uk/testing-openbim-data-exchange-for-client-specific-data-sets/, accessed on Nov. 21, 2018.

[20] Fiatech. Whitepaper FIATECH AutoCodes 2012.

[21] Lee, Y.C., Park, J., Won, J., Park, H.K., Uhm, M., Lee, Y. (2016). Can experience overcome the cognitive challenges in drawing-based design review? - Design review experiments. 16th International Conference on Construction Applications of Virtual Reality, HK, pp. 144-156. 

[22] Solihin, W. (2004). Lessons learned from experience of code-checking implementation in Singapore. Building SMART Conference, Singapore.

[23] Ayyadurai, C. (2016). Automated rule checking for in-house BIM norms of building models. Eindhoven University of Technology Department of the Build Environment Design Systems Group.

[24] LabVIEW: National Instruments. What Is LabVIEW? 2018. http://www.ni.com/en-gb/shop/labview.html, accessed on Nov. 12, 2018.

[25] Microsoft Azure. Microsoft Azure Cloud Computing Platform & Services 2018. https://azure.microsoft.com/en-us/, accessed on Nov. 12, 2018.

[26] Geometry Gym for BIM 2018. https://www.grasshopper3d.com/groups/group/show?groupUrl=geometrygym, accessed on Nov. 12, 2018.

[27] Grasshopper3d. Grasshopper Algorithmic Modeling for Rhino 2018. https://www.grasshopper3d.com/events/graph-modelling-with-grasshopper3d-introduction-to-data-flow, accessed on Nov. 12, 2018.

[28] Aerle, M.L. (2015). IFC dataflow - an open framework for connecting BIM web services. Eindhoven University of Technology Department of the Build Environment Design Systems Group.

[29] NIBS. buildingSMART alliance BIM Service interface exchange (BIMSie) Project - National Institute of Building Sciences 2015. https://www.nibs.org/page/bsa_bimsie, accessed on Nov. 11, 2018.

[30] Olsen, W. (2004). Triangulation in social research: Qualitative and quantitative methods can really be mixed. Development in Sociology.

[31] Radhakrishna, R.B., Relado, R.Z. (2012). Ensuring data quality in extension research and evaluation studies. The Journal of Extension, 50(3). 

[32] Farmer, R., Oakman, P., Rice, P. (2016). A review of free online survey tools for undergraduate students. MSOR Connections, 15(1): 71-78. https://doi.org/10.21100/msor.v15i1.311

[33] Raosoft. Sample Size Calculator 2004. http://www.raosoft.com/samplesize.html, accessed on Nov. 19, 2018.

[34] Descriptive statistics. Wikipedia 2018. https://en.wikipedia.org/w/index.php?title=Descriptive_statistics&oldid=867691828, accessed on November 15, 2018.

[35] Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education, 15: 625-632. https://doi.org/10.1007/s10459-010-9222-y

[36] Braun, V., Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2): 77-101. https://doi.org/10.1191/1478088706qp063oa

[37] NHS. WRITING AN EFFECTIVE QUESTIONNAIRE. https://www.england.nhs.uk/wp-content/uploads/2018/01/bitesize-guide-writing-an-effective-questionnaire.pdf, accessed on Aug. 18, 2020.

[38] Ross, K.N. (2005). Quantitative research methods in educational planning. UNESCO International Institute for Educational Planning.

[39] Foulkes, S.H., Bion, W. Bion and Foulkes at Northfield |Group Psychotherapy Psychotherapy. Scribd 1965. https://www.scribd.com/document/251052617/Bion-and-Foulkes-at-Northfield, accessed on Jan. 16, 2019.