Identification of early signals on social trends by web scraping and text analytics


In this project, we combine visual analysis with multiple data driven approaches (such as web scraping, text mining, network analysis, spatio-temporal analysis and so on) to analyse how the concept of smart buildings diffuses in Germany and in which topics users are highly engaged.  

Research questions#

  • How has the smart building concept diffused timely and spatially into the society?
  • What are the most popular sub-topics of the smart building concept and how are those sub-topics developing? Whether there are relationships among those popular sub-topics?
  • How are the technologies/products of smart building concept developed?
  • How about the online opinion towards smart buildings is? What are the additional benefits that come through smart buildings? What are the barriers for the development of smart buildings?


  • A series of available data sources (eg. Twitter, online news, council minutes) for social trend tracking
  • A reusable scheme for collecting large amounts of unstructured text data on a specific topic from council minutes, social media and online newspapers
  • A framework (tools and methods) for analyzing and visualizing social trends by massive unstructured text data
© Fraunhofer ISI
Analysis Framework
© Fraunhofer ISI
Workflow – Analysis of Council Meeting Minutes