We’re pleased to release version 1.0 of Social Feed Manager. This re-architected version of SFM has the functionality to select, create, manage, and explore social media collections from Twitter, Flickr, and Sina Weibo. It includes:
- A web user interface that lets users directly build, manage, and export social media collections
- Support for multiple Twitter public APIs, Flickr, and Sina Weibo
- Options for exporting the data in several formats
- Deployment using Docker and AWS
- Exploration of data using the Elasticsearch/Logstash/Kibana
- Command-line tools for working with the data as stored in WARCs
- Quick start guide for users and technical documentation.
We’re especially proud to have built into SFM an approach to collecting and storage that documents the many aspects of the creation, collection, and harvesting activity producing social media datasets. This documentation is intended to support the needs of researchers and archivists. For a deeper dive into this work, see Provenance of a Tweet, a working paper on this approach– we welcome comments.
We have plenty of work ahead to improve SFM’s usability and functionality. To understand the current limitations of SFM and known areas for future development, see our Limitations and Known Issues and tickets in GitHub. We are in the process updating our roadmap to reflect how we intend to approach further development. This work will be driven by feedback from our technical partners, users here at GW, our advisory committee, partners in the archives and research communities, and others willing to give it a whirl. We would be happy to talk with anyone who has feedback for us on any aspect, from deployment to using data. Find us on twitter, email to sfm@gwu.edu, or the sfm-dev google group.
Getting to this point in Social Feed Manager’s development was the result of the hard work of the project team, which got started under the leadership of Dan Chudnov and is supported by funding from the National Historic Records and Publications Commission.