Skip to content

Commit

Permalink
Added os extended attribute support for Linux #504 (#595)
Browse files Browse the repository at this point in the history
  • Loading branch information
joachimmetz authored Jul 29, 2021
1 parent b2ef378 commit 32d59ce
Show file tree
Hide file tree
Showing 12 changed files with 156 additions and 19 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/test_docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
- name: Install dependencies
run: |
dnf copr -y enable @gift/dev
dnf install -y python3 libbde-python3 libewf-python3 libfsapfs-python3 libfsext-python3 libfshfs-python3 libfsntfs-python3 libfsxfs-python3 libfvde-python3 libfwnt-python3 libluksde-python3 libmodi-python3 libqcow-python3 libsigscan-python3 libsmdev-python3 libsmraw-python3 libvhdi-python3 libvmdk-python3 libvsgpt-python3 libvshadow-python3 libvslvm-python3 python3-cffi python3-cryptography python3-dfdatetime python3-dtfabric python3-idna python3-mock python3-pbr python3-pytsk3 python3-pyyaml python3-setuptools python3-six
dnf install -y python3 libbde-python3 libewf-python3 libfsapfs-python3 libfsext-python3 libfshfs-python3 libfsntfs-python3 libfsxfs-python3 libfvde-python3 libfwnt-python3 libluksde-python3 libmodi-python3 libqcow-python3 libsigscan-python3 libsmdev-python3 libsmraw-python3 libvhdi-python3 libvmdk-python3 libvsgpt-python3 libvshadow-python3 libvslvm-python3 python3-cffi python3-cryptography python3-dfdatetime python3-dtfabric python3-idna python3-mock python3-pbr python3-pytsk3 python3-pyxattr python3-pyyaml python3-setuptools python3-six
- name: Run tests
env:
LANG: C.utf8
Expand Down Expand Up @@ -57,7 +57,7 @@ jobs:
run: |
add-apt-repository -y ppa:gift/dev
apt-get update -q
apt-get install -y python3 libbde-python3 libewf-python3 libfsapfs-python3 libfsext-python3 libfshfs-python3 libfsntfs-python3 libfsxfs-python3 libfvde-python3 libfwnt-python3 libluksde-python3 libmodi-python3 libqcow-python3 libsigscan-python3 libsmdev-python3 libsmraw-python3 libvhdi-python3 libvmdk-python3 libvsgpt-python3 libvshadow-python3 libvslvm-python3 python3-cffi-backend python3-cryptography python3-dfdatetime python3-distutils python3-dtfabric python3-idna python3-mock python3-pbr python3-pytsk3 python3-setuptools python3-six python3-yaml
apt-get install -y python3 libbde-python3 libewf-python3 libfsapfs-python3 libfsext-python3 libfshfs-python3 libfsntfs-python3 libfsxfs-python3 libfvde-python3 libfwnt-python3 libluksde-python3 libmodi-python3 libqcow-python3 libsigscan-python3 libsmdev-python3 libsmraw-python3 libvhdi-python3 libvmdk-python3 libvsgpt-python3 libvshadow-python3 libvslvm-python3 python3-cffi-backend python3-cryptography python3-dfdatetime python3-distutils python3-dtfabric python3-idna python3-mock python3-pbr python3-pytsk3 python3-pyxattr python3-setuptools python3-six python3-yaml
- name: Run tests
env:
LANG: en_US.UTF-8
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test_tox.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
add-apt-repository -y ppa:deadsnakes/ppa
add-apt-repository -y ppa:gift/dev
apt-get update -q
apt-get install -y build-essential git libffi-dev python${{ matrix.python-version }} python${{ matrix.python-version }}-dev python${{ matrix.python-version }}-venv libbde-python3 libewf-python3 libfsapfs-python3 libfsext-python3 libfshfs-python3 libfsntfs-python3 libfsxfs-python3 libfvde-python3 libfwnt-python3 libluksde-python3 libmodi-python3 libqcow-python3 libsigscan-python3 libsmdev-python3 libsmraw-python3 libvhdi-python3 libvmdk-python3 libvsgpt-python3 libvshadow-python3 libvslvm-python3 python3-cffi-backend python3-cryptography python3-dfdatetime python3-distutils python3-dtfabric python3-idna python3-mock python3-pbr python3-pip python3-pytsk3 python3-setuptools python3-six python3-yaml
apt-get install -y build-essential git libffi-dev python${{ matrix.python-version }} python${{ matrix.python-version }}-dev python${{ matrix.python-version }}-venv libbde-python3 libewf-python3 libfsapfs-python3 libfsext-python3 libfshfs-python3 libfsntfs-python3 libfsxfs-python3 libfvde-python3 libfwnt-python3 libluksde-python3 libmodi-python3 libqcow-python3 libsigscan-python3 libsmdev-python3 libsmraw-python3 libvhdi-python3 libvmdk-python3 libvsgpt-python3 libvshadow-python3 libvslvm-python3 python3-cffi-backend python3-cryptography python3-dfdatetime python3-distutils python3-dtfabric python3-idna python3-mock python3-pbr python3-pip python3-pytsk3 python3-pyxattr python3-setuptools python3-six python3-yaml
- name: Install tox
run: |
python3 -m pip install tox
Expand Down
2 changes: 1 addition & 1 deletion config/appveyor/install.ps1
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Script to set up tests on AppVeyor Windows.

$Dependencies = "PyYAML cffi cryptography dfdatetime dtfabric idna libbde libewf libfsapfs libfsext libfshfs libfsntfs libfsxfs libfvde libfwnt libluksde libmodi libqcow libsigscan libsmdev libsmraw libvhdi libvmdk libvsgpt libvshadow libvslvm mock pbr pytsk3 six"
$Dependencies = "PyYAML cffi cryptography dfdatetime dtfabric idna libbde libewf libfsapfs libfsext libfshfs libfsntfs libfsxfs libfvde libfwnt libluksde libmodi libqcow libsigscan libsmdev libsmraw libvhdi libvmdk libvsgpt libvshadow libvslvm mock pbr pytsk3 six xattr"
$Dependencies = ${Dependencies} -split " "

$Output = Invoke-Expression -Command "git clone https://github.com/log2timeline/l2tdevtools.git ..\l2tdevtools 2>&1"
Expand Down
2 changes: 1 addition & 1 deletion config/dpkg/control
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Homepage: https://github.com/log2timeline/dfvfs

Package: python3-dfvfs
Architecture: all
Depends: libbde-python3 (>= 20140531), libewf-python3 (>= 20131210), libfsapfs-python3 (>= 20201107), libfsext-python3 (>= 20210721), libfshfs-python3 (>= 20210722), libfsntfs-python3 (>= 20200921), libfsxfs-python3 (>= 20210726), libfvde-python3 (>= 20160719), libfwnt-python3 (>= 20210717), libluksde-python3 (>= 20200101), libmodi-python3 (>= 20210405), libqcow-python3 (>= 20201213), libsigscan-python3 (>= 20191221), libsmdev-python3 (>= 20140529), libsmraw-python3 (>= 20140612), libvhdi-python3 (>= 20201014), libvmdk-python3 (>= 20140421), libvsgpt-python3 (>= 20210207), libvshadow-python3 (>= 20160109), libvslvm-python3 (>= 20160109), python3-cffi-backend (>= 1.9.1), python3-cryptography (>= 2.0.2), python3-dfdatetime (>= 20210509), python3-dtfabric (>= 20170524), python3-idna (>= 2.5), python3-pytsk3 (>= 20210419), python3-yaml (>= 3.10), ${python3:Depends}, ${misc:Depends}
Depends: libbde-python3 (>= 20140531), libewf-python3 (>= 20131210), libfsapfs-python3 (>= 20201107), libfsext-python3 (>= 20210721), libfshfs-python3 (>= 20210722), libfsntfs-python3 (>= 20200921), libfsxfs-python3 (>= 20210726), libfvde-python3 (>= 20160719), libfwnt-python3 (>= 20210717), libluksde-python3 (>= 20200101), libmodi-python3 (>= 20210405), libqcow-python3 (>= 20201213), libsigscan-python3 (>= 20191221), libsmdev-python3 (>= 20140529), libsmraw-python3 (>= 20140612), libvhdi-python3 (>= 20201014), libvmdk-python3 (>= 20140421), libvsgpt-python3 (>= 20210207), libvshadow-python3 (>= 20160109), libvslvm-python3 (>= 20160109), python3-cffi-backend (>= 1.9.1), python3-cryptography (>= 2.0.2), python3-dfdatetime (>= 20210509), python3-dtfabric (>= 20170524), python3-idna (>= 2.5), python3-pytsk3 (>= 20210419), python3-pyxattr (>= 0.7.2), python3-yaml (>= 3.10), ${python3:Depends}, ${misc:Depends}
Description: Python 3 module of dfVFS
dfVFS, or Digital Forensics Virtual File System, provides read-only access to
file-system objects from various storage media types and file formats. The goal
Expand Down
8 changes: 8 additions & 0 deletions dependencies.ini
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,14 @@ pypi_name: libvslvm-python
rpm_name: libvslvm-python3
version_property: get_version()

[xattr]
dpkg_name: python3-pyxattr
is_optional: true
minimum_version: 0.7.2
pypi_name: pyxattr
rpm_name: python3-pyxattr
version_property: __version__

[yaml]
dpkg_name: python3-yaml
l2tbinaries_name: PyYAML
Expand Down
2 changes: 1 addition & 1 deletion dfvfs/file_io/os_file_io.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@


class OSFile(file_io.FileIO):
"""File input/output (IO) object using os."""
"""File input/output (IO) object that uses the operating system."""

def __init__(self, resolver_context, path_spec):
"""Initializes a file input/output (IO) object.
Expand Down
116 changes: 116 additions & 0 deletions dfvfs/vfs/os_attribute.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# -*- coding: utf-8 -*-
"""The operating system attribute implementation."""

import os

try:
import xattr
except ImportError:
xattr = None

from dfvfs.vfs import attribute


class OSExtendedAttribute(attribute.Attribute):
"""Extended attribute that uses the operating system."""

def __init__(self, location, name):
"""Initializes an attribute.
Args:
location (str): path of the file.
name (str): name of the extended attribute.
"""
super(OSExtendedAttribute, self).__init__()
self._current_offset = 0
self._data = xattr.getxattr(location, name)
self._location = location
self._name = name
self._size = len(self._data)

@property
def name(self):
"""str: name."""
return self._name

# Note: that the following functions do not follow the style guide
# because they are part of the file-like object interface.
# pylint: disable=invalid-name

def read(self, size=None):
"""Reads a byte string from the file input/output (IO) object.
The function will read a byte string of the specified size or
all of the remaining data if no size was specified.
Args:
size (Optional[int]): number of bytes to read, where None is all
remaining data.
Returns:
bytes: data read.
Raises:
IOError: if the read failed.
OSError: if the read failed.
"""
if size is None:
size = self._size

data = self._data[self._current_offset:self._current_offset + size]
self._current_offset += len(data)
return data

def seek(self, offset, whence=os.SEEK_SET):
"""Seeks to an offset within the file input/output (IO) object.
Args:
offset (int): offset to seek.
whence (Optional[int]): value that indicates whether offset is an
absolute or relative position within the file.
Raises:
IOError: if the seek failed.
OSError: if the seek failed.
"""
if whence == os.SEEK_CUR:
offset += self._current_offset
elif whence == os.SEEK_END:
offset = self._size - offset
elif whence != os.SEEK_SET:
raise IOError('Invalid whence')

if offset < 0:
raise IOError('Invalid offset')

self._current_offset = offset

# get_offset() is preferred above tell() by the libbfio layer used in libyal.
def get_offset(self):
"""Retrieves the current offset into the file input/output (IO) object.
Returns:
int: current offset into the file input/output (IO) object.
"""
return self._current_offset

# Pythonesque alias for get_offset().
def tell(self):
"""Retrieves the current offset into the file input/output (IO) object."""
return self.get_offset()

def get_size(self):
"""Retrieves the size of the file input/output (IO) object.
Returns:
int: size of the file input/output (IO) object.
"""
return self._size

def seekable(self):
"""Determines if a file input/output (IO) object is seekable.
Returns:
bool: True since a file IO object provides a seek method.
"""
return True
30 changes: 20 additions & 10 deletions dfvfs/vfs/os_file_entry.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,23 @@

import pysmdev

try:
import xattr
except ImportError:
xattr = None

from dfdatetime import posix_time as dfdatetime_posix_time

from dfvfs.lib import definitions
from dfvfs.lib import errors
from dfvfs.path import os_path_spec
from dfvfs.vfs import attribute
from dfvfs.vfs import file_entry
from dfvfs.vfs import os_attribute


class OSDirectory(file_entry.Directory):
"""File system directory that uses os."""
"""File system directory that uses the operating system."""

def _EntriesGenerator(self):
"""Retrieves directory entries.
Expand Down Expand Up @@ -99,6 +105,7 @@ def __init__(self, resolver_context, file_system, path_spec, is_root=False):
resolver_context, file_system, path_spec, is_root=is_root,
is_virtual=False)
self._is_windows_device = is_windows_device
self._location = location
self._name = None
self._stat_info = stat_info

Expand Down Expand Up @@ -140,6 +147,12 @@ def _GetAttributes(self):
stat_attribute = self._GetStatAttribute()
self._attributes = [stat_attribute]

if xattr:
for name in xattr.listxattr(self._location):
extended_attribute = os_attribute.OSExtendedAttribute(
self._location, name)
self._attributes.append(extended_attribute)

return self._attributes

def _GetDirectory(self):
Expand All @@ -162,11 +175,10 @@ def _GetLink(self):
if self._link is None:
self._link = ''

location = getattr(self.path_spec, 'location', None)
if location is None:
if self._location is None:
return self._link

self._link = os.readlink(location)
self._link = os.readlink(self._location)
self._link = os.path.abspath(self._link)

return self._link
Expand Down Expand Up @@ -291,9 +303,8 @@ def modification_time(self):
def name(self):
"""str: name of the file entry, without the full path."""
if self._name is None:
location = getattr(self.path_spec, 'location', None)
if location is not None:
self._name = self._file_system.BasenamePath(location)
if self._location is not None:
self._name = self._file_system.BasenamePath(self._location)
return self._name

@property
Expand Down Expand Up @@ -323,11 +334,10 @@ def GetParentFileEntry(self):
Returns:
OSFileEntry: parent file entry or None if not available.
"""
location = getattr(self.path_spec, 'location', None)
if location is None:
if self._location is None:
return None

parent_location = self._file_system.DirnamePath(location)
parent_location = self._file_system.DirnamePath(self._location)
if parent_location is None:
return None

Expand Down
4 changes: 2 additions & 2 deletions dfvfs/vfs/os_file_system.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
"""The file system implementation that is provided by the operating system."""
"""The operating system file system implementation."""

import os
import platform
Expand All @@ -14,7 +14,7 @@


class OSFileSystem(file_system.FileSystem):
"""File system provided by the operating system."""
"""File system that uses the operating system."""

if platform.system() == 'Windows':
PATH_SEPARATOR = '\\'
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,4 @@ libvsgpt-python >= 20210207
libvshadow-python >= 20160109
libvslvm-python >= 20160109
pytsk3 >= 20210419
pyxattr >= 0.7.2
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ requires = libbde-python3 >= 20140531
python3-dtfabric >= 20170524
python3-idna >= 2.5
python3-pytsk3 >= 20210419
python3-pyxattr >= 0.7.2
python3-pyyaml >= 3.10

[bdist_wheel]
Expand Down
3 changes: 2 additions & 1 deletion tests/vfs/os_file_entry.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,8 @@ def testGetAttributes(self):

file_entry._GetAttributes()
self.assertIsNotNone(file_entry._attributes)
self.assertEqual(len(file_entry._attributes), 1)
# Note that on some platforms this file can have extended attributes.
self.assertGreaterEqual(len(file_entry._attributes), 1)

test_attribute = file_entry._attributes[0]
self.assertIsInstance(test_attribute, attribute.StatAttribute)
Expand Down

0 comments on commit 32d59ce

Please sign in to comment.