Systematic and Complex Reviews

Most articles will list a corresponding author. Email the authors of the articles included in your final data analysis group. Let them know the nature of your project and ask them if they can provide you with any leads to other articles, reports, studies or unpublished material that they think should be included in the analysis. If the list is long, you can prioritize which authors you contact, perhaps the most cited? Alternatively, you can brainstorm within your team to identify any key researchers not already identified and contact them in a similar way. Keep a record of the sources searched and the resources located.

A lot of research material is conducted by PhD students and is contained in theses and dissertations. Theses can be located in special databases such as:

• EBSCO Open Dissertations
• Open Access Theses and Dissertations
• Networked Digital Library of Theses and Dissertations
• OAIster
• ProQuest Dissertations & Theses
• Trove
• EThOS

Many universities will also have their own repositories available for searching e.g. ResearchDirect. Keep a record of the sources searched and the resources located.

Grey material

Grey material is the most common form of supplementary searching that is reported in systematic reviews and scoping reviews. Grey material is any material that is published by organizations whose primary role is not publication, an example is a government report. There are many different types of grey material that will be relevant to projects or reviews e.g. clinical trials, organizational papers and reports, position statements, and parliamentary papers to name a few. Most of this material will be published on the internet. Here are some sources to locate this type of material.

Clinical trials

Locating unpublished clinical trials is essential for any kind of effectiveness review. There are a number of different directories you can search to locate this kind of material.

• Australian New Zealand Clinical Trials Registry (ANZCTR) – ANZCTR is a register of clinical trials being undertaken in Australia and New Zealand.
• Cochrane Central Register of Controlled Trials (CENTRAL) – Access via Cochrane Library, a high quality registry of randomized and quasi-randomized trials.
• ClinicalTrials.gov – a registry of public and private international clinical trials.
• WHO's International Clinical Trials Registry Platform – a voluntary platform of clinical trials, searchable from International Clinical Trials Registry Platform (ICTRP) search portal
• https://www.australianclinicaltrials.gov.au/clinical-trial-registries

Organisational websites

• Brainstorm a list of organizations relevant to the topic of the project or review. Locate their webpage. Use the search function of the site and keywords to locate resources. Alternatively look for relevant tabs on the web page to locate material for example: Research or Publications. This is sometimes called crawling the site.

Domain searching

• Generally specific types of organizations are allocated specific types of URLs domain names. ICANN (2013) is the not-for-profit organization coordinating the allocation and assignment of these Internet’s unique identifiers. Some examples include:

  site:.org      an organizational or not for profit site (generally) site:.asn      an associations site

  site:.gov      a government site

  site:.edu      an educational site, site:.ac in the UK

  Using this structure, it is possible to filter the grey material on the internet to locate more relevant information for a project or review. As with Google Scholar searches, multiple simpler and separate consecutive searches may be needed.

  Add site:domain to the web browser search string to restrict the search to those sites only. You can also extend the domain name to refine for country e.g. site:.gov.au for Australian government, or site:.nsw.gov.au for state government sites.

Grey material repositories

• You may be able to locate grey material via Open access repositories. An open access repository is a searchable collection of resources available via online databases accessed through the internet. They may provide direct access for links and information about where this material may be stored. A University Repository is an example of a resource that may provide access to grey material for example thesis or reports written by research students and academic staff.

Other examples of grey material repositories include:

• OpenDOAR: Directory of Open Access Repositories
• IRIS: Institutional Repository for information sharing
• APO: Analysis & Policy Observatory
• RAND Publication Database
• Base
• Openaire

Ask an expert in the field or your stakeholders or supervisors for any recommendations of relevant articles or unpublished material. Keep a record of the sources searched and the resources located.

Google Scholar is not a database that is recommended for a systematic search, however it can be very useful when conducting a supplementary search. Use Google Scholar to locate theses, reports, or conference papers relevant to your project. Google Scholar searches will need to use multiple, separate simpler search strings in consecutive searches. Truncation and proximity searching work in different ways to databases and wildcards cannot be applied. Here is a list of other limitations with a Google Scholar search.

• Google Scholar has a 256-character limit to its search string. Any terms beyond that will be deleted from the search.
• Google Scholar does not use truncation. It uses automatic stemming. Automatic stemming identifies the ‘root’ of the keywords used, automatically searching for any additional letters. For example, the keyword 'learn' retrieves ‘learner’, ‘learning’, ‘learns’, ‘learnable’. But the keyword 'sociology' would not retrieve related keywords such as ‘sociologist’, ‘sociological’ as these synonyms do not contain the stem word 'sociologist '. Also stemming only works for complete words i.e. 'sociologist' rather than ‘sociolo’. This feature can be turned off by putting a '+' symbol in front of the keyword. For example: +learn will only return records that contain learn (6,870,000 results) rather than learn (6,910,000 results), this total would include records with 'learn', ‘learner’, ‘learning’, ‘learns’ and ‘learnable’.
• Using the asterisk for truncation in Google Scholar returns whole word results rather than alternative endings e.g. road* will return roadway but not roadie.
• Wildcard for variations in spelling do not work in Google Scholar.
• There are some commands you can use to narrow the search e.g.: intitle:, allintitle:, allinurl:, intext:, allintext:, AROUND(X). You can find more info here https://ahrefs.com/blog/google- advanced-search-operators/
• You can also find some ways to limit a Google Scholar search in the Advanced Search option. Use with caution, these limits are extremely basic.

The amount of material that Google Scholar can locate is overwhelming and much of it will be irrelevant. Prior to searching, decide in your team on some limits. e.g., Decide how many different simple search strings you will use, set page number results to a specific number, 20, 50? Then decide how many pages you are prepared to review, 1, 2, 3? How many you decide will depend on the  resources in your team and the expectation or scope of the project. Keep a record of the sources searched and the resources located.