Skip to content

DataSet File Format

DSFF logo

DSFF (DataSet File Format) is a tiny library relying on openpyxl that stores a dataset with its features, for use with machine learning, in an XSLX file whose structure is enforced. It makes it easy to store, edit and exchange a dataset.

PyPi Python Versions Build Status Security License

It is used with the Packing Box to export datasets in a convenient format.

Features

  • A single .dsff file holds the data, the description of each feature and the metadata of the dataset.
  • The file is a regular XSLX workbook: it opens in any spreadsheet editor.
  • Conversion from and to ARFF (for Weka), CSV, SQLite databases and FilelessDataset folders from the Packing Box.
  • With pyarrow, conversion from and to the Apache Arrow formats Feather, ORC and Parquet.

Quick start

$ pip install dsff

>>> import dsff
>>> with dsff.DSFF("my-dataset", "w") as f:
...     f.write(data=[["name", "size", "label"], ["a.exe", 1024, "upx"], ["b.exe", 2048, "?"]],
...             features={'size': "size of the executable"},
...             metadata={'source': "my lab"})
...     f.to_csv()  # creates ./my-dataset.csv
...
# while leaving the context, ./my-dataset.dsff is created

See Installation and Usage to go further.