notesdir package

Submodules

notesdir.api module

Provides the main entry point for using the library, Notesdir

exception notesdir.api.Error

Bases: Exception

class notesdir.api.Notesdir(conf: notesdir.conf.NotesdirConf)

Bases: object

Main entry point for working programmatically with your collection of notes.

Generally, you should get an instance using the Notesdir.for_user() method. Call close() when you’re done with it, or else use it as a context manager.

This class contains various methods such as Notesdir.move() and Notesdir.standardize() for performing high-level operations. The repo attribute, which is an instance of notesdir.repos.base.Repo, provides additional operations, some lower-level.

conf: notesdir.conf.NotesdirConf

Typically loaded from the variable conf in the file ~/.notesdir.conf.py

repo: notesdir.repos.base.Repo

Here’s an example of how to use this class. This would add the tag “personal” to every note tagged with “journal”.

from notesdir.api import Notesdir
with Notesdir.for_user() as nd:
    infos = nd.repo.query('tag:journal', 'path')
    nd.change({info.path for info in infos}, add_tags={'personal'})
backfill()

Finds all files missing title or created metadata, and attempts to set that metadata.

Missing titles are set to the filename, minus the file extension. Missing created dates are set based on the birthtime or ctime of the file.

Returns a list of all successfully changed files, and a list of exceptions encountered for other files.

change(paths: Set[str], add_tags: Set[str] = {}, del_tags: Set[str] = {}, title: Optional[str] = None, created: Optional[datetime.datetime] = None) → None

Applies all the specified changes to the specified paths.

This is a convenience method that wraps notesdir.repos.base.Repo.change()

close()

Closes the associated repo and releases any other resources.

static for_user() → notesdir.api.Notesdir

Creates an instance using the user’s ~/.notesdir.conf.py file.

Raises Exception if it does not exist or does not define configuration.

move(moves: Dict[str, str], *, into_dirs=True, check_exists=True, create_parents=False, delete_empty_parents=False) → Dict[str, str]

Moves files/directories and updates references to/from them appropriately.

moves is a dict where the keys are source paths that should be moved, and the values are the destinations. If a destination is a directory and into_dirs is True, the source will be moved into it, using the source’s filename; otherwise, the source is renamed to the destination.

This method tries not to overwrite files; if a destination path already exists, a shortened UUID will be appended to the path. You can disable that behavior by setting check_exists=False.

It’s OK for a path to occur as a key and also another key’s value. For example, {'foo': 'bar', 'bar': 'foo'} will swap the two files.

If create_parents is True, any directories in a destination path that do not yet exist will be created.

If delete_empty_parents is True, after moving files out of a directory, if the directory or any of its parent directories are empty, they will be deleted. (The root folder or current working directory will not be deleted regardless.)

Returns a dict mapping paths of files that were moved, to their final paths.

new(template_name: str, dest: Optional[str] = None) → str

Creates a new file using the specified template.

The template name will be looked up using template_for-name().

Raises FileNotFoundError if the template cannot be found.

If dest is not given, a target file name will be generated.

The following names are defined in the template’s namespace:

Returns the path of the created file.

organize() → Dict[str, str]

Reorganizes files using the function set in notesdir.conf.NotesdirConf.path_organizer.

For every file in your note directories (defined by notesdir.conf.RepoConf.root_paths), this method will call that function with the file’s FileInfo, and move the file to the path the function returns.

Note that the function will only be called for files, not directories. You cannot directly move a directory by this method, but you can effectively move one by moving all the files from it to the same new directory.

This method deletes any empty directories that result from the moves it makes, and creates any directories it needs to.

The FileInfo is retrieved using notesdir.models.FileInfoReq.full().

replace_path_hrefs(original: str, replacement: str) → None

Finds and replaces links to the original path with links to the new path.

Note that this does not currently replace links to children of the original path - e.g., if original is “/foo/bar”, a link to “/foo/bar/baz” will not be updated.

No files are moved, and this method does not care whether or not the original or replacement paths refer to actual files.

template_for_name(name: str) → Optional[str]

Returns the path to the template for the given name, if one is found.

If treating the name as a relative or absolute path leads to a file, that file is used. Otherwise, the name is looked up from Notesdir.templates_by_name(), case-insensitively. Returns None if a matching template cannot be found.

templates_by_name() → Dict[str, str]

Returns paths of note templates that are known based on the config.

The name is the part of the filename before any . character. If multiple templates have the same name, the one whose path is lexicographically first will appear in the dict.

notesdir.cli module

Command-line interface for notesdir.

notesdir.cli.argparser() → argparse.ArgumentParser
notesdir.cli.main(args=None) → int

Runs the tool and returns its exit code.

args may be an array of string command-line arguments; if absent, the process’s arguments are used.

notesdir.conf module

class notesdir.conf.DirectRepoConf(root_paths: Set[str], ignore: Callable[[str, str], bool] = <function default_ignore>, skip_parse: Callable[[str, str], bool] = <function default_skip_parse>, preview_mode: bool = False)

Bases: notesdir.conf.RepoConf

Configures notesdir to access notes without caching, via notesdir.repos.DirectRepo.

instantiate()
root_paths: Set[str]

The folders that should be searched (recursively) when querying for notes, finding backlinks, etc.

Must not be empty.

class notesdir.conf.NotesdirConf(repo_conf: 'RepoConf', template_globs: 'Set[str]' = <factory>, path_organizer: 'Callable[[FileInfo], str]' = <function NotesdirConf.<lambda> at 0x1050f3dc0>, cli_path_output_rewriter: 'Callable[[str], str]' = <function NotesdirConf.<lambda> at 0x1050f3ca0>)

Bases: object

cli_path_output_rewriter()

Changes how paths are printed to the console by the CLI.

By default paths are printed in their absolute, canonical form. This setting lets you change that by defining a function which takes the absolute path as a parameter and returns the path that should be printed.

My use case for this is that I store some notes in iCloud, and the absolute path to the iCloud folder contains a space, which means command-clicking the path in the terminal does not work. By creating a symlink and rewriting the paths to use the symlink before printing them, I can ensure the paths are clickable.

conf.cli_path_output_rewriter = lambda path: path.replace('/Users/jacob/Library/Mobile Documents/com~apple~CloudDocs/', '/Users/jacob/iCloud/')
classmethod for_user() → notesdir.conf.NotesdirConf
instantiate()
path_organizer()

Defines the rule for rewriting paths used by the organize command and notesdir.api.Notesdir.organize().

You can use this to standardize filenames or to organize your files via tags, date, or other criteria.

For example, the following converts all filenames to lowercase versions of the note title (if any):

import os.path
def path_organizer(info):
    dirname, filename = os.path.split(info.path)
    suffix = os.path.splitext(filename)[1]
    if info.title:
        return os.path.join(dirname, into.title.lower() + suffix)
    return os.path.join(dirname, filename.lower())

conf.path_organizer = path_organizer

Here’s an example of organizing by important tags:

import os.path
def path_organizer(info):
    for tag in ['secrets', 'journal', 'grievances']:
        if tag in info.tags:
            return f'/Users/jacob/notes/{tag}/{os.path.basename(info.path)}'
    return f'/Users/jacob/notes/misc/{os.path.basename(info.path)}'

conf.path_organizer = path_organizer

Some helper functions are provided for use in path organizers:

repo_conf: notesdir.conf.RepoConf

Configures how to access your collection of notes.

standardize()
template_globs: Set[str]

A set of path globs such as {"/notes/templates/*.mako"} to search for templates.

This is used for the CLI command new, and template-related methods of notesdir.api.Notesdir.

class notesdir.conf.RepoConf(root_paths: Set[str], ignore: Callable[[str, str], bool] = <function default_ignore>, skip_parse: Callable[[str, str], bool] = <function default_skip_parse>, preview_mode: bool = False)

Bases: object

Base class for repo config. Use a subclass such as SqliteRepoConf.

ignore(filename: str) → bool

Use this to indicate files or folders that should not be processed by notesdir at all.

The first argument is the path to the directory containing the file/folder, and the second argument is the filename.

If this function returns True for a given path, neither that path nor any of its child paths will be parsed, or returned in any queries, or affected by the organize command.

The mv command may still move these files when explicitly instructed to do so or when moving a directory containing them, and the info command will still show backlinks from other (non-ignored) files.

The current default behavior is to ignore all files or folders whose name begins with a period (.), and also .icloud files.

instantiate()
preview_mode: bool = False

If True, commands that would change notes should instead just print a list of changes to the console.

Instead of setting this in your .notesdir.conf.py, you can pass a --preview command-line argument to relevant commands.

root_paths: Set[str]

The folders that should be searched (recursively) when querying for notes, finding backlinks, etc.

Must not be empty.

skip_parse(filename: str) → bool

Use this to indicate files or folders that should not be parsed (or edited) by notesdir.

The first argument is the path to the directory containing the file/folder, and the second argument is the filename.

If this function returns True for a given path, parsing will be skipped for both it and its child paths. For such files, only the path and backlinks attributes will be populated on notesdir.models.FileInfo.

Unlike notesdir.conf.RepoConf.ignore, unparsed files are still potentially returned in queries and affected by the organize command. Note that parsing is automatically skipped for ignored files.

The current default behavior does not skip anything.

standardize()
class notesdir.conf.SqliteRepoConf(root_paths: Set[str], ignore: Callable[[str, str], bool] = <function default_ignore>, skip_parse: Callable[[str, str], bool] = <function default_skip_parse>, preview_mode: bool = False, cache_path: Optional[str] = None)

Bases: notesdir.conf.DirectRepoConf

Configures notesdir to access notes with caching, via notesdir.repos.SqliteRepo.

cache_path: str = None

Required. Path where the SQLite database file should be stored.

The file will be created if it does not exist. The file is only a cache; you can safely delete it when the tool is not running, though you will then have to wait for the cache to be rebuilt the next time you run the tool.

instantiate()
notesdir.conf.default_ignore(parentpath: str, filename: str) → bool
notesdir.conf.default_skip_parse(parentpath: str, filename: str) → bool
notesdir.conf.resource_path_fn(path: str) → Optional[notesdir.models.DependentPathFn]

Enables moving files in .resources directories when the owner moves.

This is meant for use in a NotesdirConf.path_organizer if you’d like to follow the convention of putting attachments for a file in a directory next to the file, with a suffix of .resources. For example, an attachment cat.png for the file /notes/foo.md would be at /notes/foo.md.resources/cat.png.

This function lets you ensure that if foo.md is renamed by your organizer, foo.md.resources will be too.

Example usage:

def my_path_organizer(info):
    rewrite = resource_path_fn(info.path)
    if rewrite:
        return rewrite
    # put the rest of your organizer rules here
conf.path_organizer = my_path_organizer
notesdir.conf.rewrite_name_using_title(info: notesdir.models.FileInfo) → str

If the given info has a title, returns an updated path using that title.

The following adjustments are made:

  • Title is truncated to 60 characters

  • Characters are converted to lowercase

  • Only the letters a-z and digits 0-9 are kept; all other characters are replaced with dashes

  • Consecutive dashes are collapsed to a single dash

  • Leading and trailing dashes are removed

For example, for a file at /foo/bar.md with title “Everything is awful”, the path returned would be /foo/everything-is-awful.md.

The file extension from the original path is kept, but note that currently this will not work properly for files with multiple extensions (eg tar.gz). That shouldn’t be an issue right now since none of the file types for which title metadata is supported typically use multiple extensions.

If there is no title, the path is returned unchanged.

This is meant to be used as, or as part of, a NotesdirConf.path_organizer.

notesdir.models module

Defines classes for representing note details, queries, and update requests.

The most important classes are FileInfo , FileEditCmd , and FileQuery

class notesdir.models.AddTagCmd(path: str, value: str)

Bases: notesdir.models.FileEditCmd

Represents a request to add a tag to a document.

If the document already contains the tag, this request should be treated as a no-op.

value: str

The tag to add.

class notesdir.models.CreateCmd(path: str, contents: str)

Bases: notesdir.models.FileEditCmd

Represents a request to create a new file.

contents: str
class notesdir.models.DelTagCmd(path: str, value: str)

Bases: notesdir.models.FileEditCmd

Represents a request to remove a tag from a document.

If the document does not contain the tag, this request should be treated as a no-op.

value: str

The tag to remove.

class notesdir.models.DependentPathFn(determinant: str, fn: Callable[[notesdir.models.FileInfo], str])

Bases: object

Indicates that a path can be calculated based on the FileInfo for another path.

You can return this from a notesdir.conf.NotesdirConf.path_organizer when one file’s path depends on another’s. notesdir.api.Notesdir.organize() will call the given fn with the info for the path specified by determinant, but the path in the info will reflect any pending move for that file (even if they have not been executed in the filesystem yet).

determinant: str
fn: Callable[[notesdir.models.FileInfo], str]
class notesdir.models.FileEditCmd(path: str)

Bases: object

Base class for requests to make changes to a file.

path: str

Path to the file or folder that should be changed.

class notesdir.models.FileInfo(path: str, links: List[notesdir.models.LinkInfo] = <factory>, tags: Set[str] = <factory>, title: Optional[str] = None, created: Optional[datetime.datetime] = None, backlinks: List[notesdir.models.LinkInfo] = <factory>)

Bases: object

Container for the details Notesdir can parse or calculate about a file or folder.

A FileInfo instance does not imply that its path actually exists - instances may be created for nonexistent paths that have just the path and backlinks attributes filled in.

When you retrieve instances of this from methods like notesdir.repos.base.Repo.info(), which fields are populated depends on which fields you request via the FileInfoReq, as well as what fields are supported for the file type and what data is populated in the particular file.

as_json() → dict

Returns a dict representing the instance, suitable for serializing as json.

Links from other files to this file.

created: Optional[datetime.datetime] = None

The creation date of the document, according to metadata within the document, if any.

This will not automatically be populated with timestamps from the filesystem, but see guess_created().

guess_created() → Optional[datetime.datetime]

Returns the first available of: created, or the file’s birthtime, or the file’s ctime.

Returns None for paths that don’t exist.

Links from this file to other files or resources.

path: str

The resolved, absolute path for which this information applies.

tags: Set[str]

Tags for the file (e.g. “journal” or “project-idea”).

title: Optional[str] = None

The title of the document, if any.

class notesdir.models.FileInfoReq(path: bool = False, links: bool = False, tags: bool = False, title: bool = False, created: bool = False, backlinks: bool = False)

Bases: object

Allows you to specify which attributes you want when loading or querying for files.

For each attribute of FileInfo, there is a corresponding boolean attribute here, which you should set to True to indicate that you want that attribute.

Some methods that take a FileInfoReq parameter also accept strings or lists of strings as a convenience, which they will pass to parse().

created: bool = False
classmethod full() → notesdir.models.FileInfoReq

Returns an instance that requests everything.

classmethod internal() → notesdir.models.FileInfoReq

Returns an instance that requests everything which can be determined by looking at a file in isolation.

Currently this means everything except backlinks.

classmethod parse(val: Union[str, Iterable[str], notesdir.models.FileInfoReq]) → notesdir.models.FileInfoReq

Converts the parameter to a FileInfoReq, if it isn’t one already.

You can pass a comma-separated string like "path,backlinks" or a list of strings like ['path', 'backlinks']. Each listed field will be set to True in the resulting FileInfoReq.

path: bool = False
tags: bool = False
title: bool = False
class notesdir.models.FileQuery(include_tags: Set[str] = <factory>, exclude_tags: Set[str] = <factory>, sort_by: List[notesdir.models.FileQuerySort] = <factory>)

Bases: object

Represents criteria for searching for notes.

Some methods that take a FileQuery parameter also accept strings as a convenience, which they pass to parse()

If multiple criteria are specified, the query should only return notes that satisfy all the criteria.

apply_filtering(infos: Iterable[notesdir.models.FileInfo]) → Iterator[notesdir.models.FileInfo]

Yields the entries from the given iterable which match the criteria of this query.

apply_sorting(infos: Iterable[notesdir.models.FileInfo]) → List[notesdir.models.FileInfo]

Returns a copy of the given file info collection sorted using this query’s sort_by.

exclude_tags: Set[str]

If non-empty, the query should only return files that have none of the specified tags.

include_tags: Set[str]

If non-empty, the query should only return files that have all of the specified tags.

classmethod parse(strquery: Union[str, notesdir.models.FileQuery]) → notesdir.models.FileQuery

Converts the parameter to a FileQuery, if it isn’t one already.

Query strings are split on spaces. Each part can be one of the following:

  • tag:TAG1,TAG2 - notes must include all the specified tags

  • -tag:TAG1,TAG2 - notes must not include any of the specified tags

  • sort:FIELD1,FIELD2 - sort by the given fields

    • fields on the left take higher priority, e.g. sort:created,title sorts by created date first

    • a minus sign in front of a field name indicates to sort descending, e.g. sort:-backlinks or sort:filename,-created

    • supported fields: backlinks (count), created, filename, tags (count) title, path

Examples:

  • "tag:journal,food -tag:personal" - notes that are tagged both “journal” and “food” but not “personal”

sort_by: List[notesdir.models.FileQuerySort]

Indicates how to sort the results.

For example, [(FileQuerySort.BACKLINKS_COUNT, Order.DESC), (FileQuerySort.FILENAME, Order.ASC)] would sort the results so that the most-linked-to files appear first; files with equal numbers of backlinks would be sorted lexicographically.

class notesdir.models.FileQuerySort(field: FileQuerySortField, reverse: bool = False, ignore_case: bool = True, missing_first: bool = False)

Bases: object

field: notesdir.models.FileQuerySortField
ignore_case: bool = True

If True, strings are sorted as if they were lower case.

key(info: notesdir.models.FileInfo) → Union[str, int, datetime.datetime]

Returns sort key for the given file info for the field specified in this instance.

This is affected by the values of ignore_case and missing_first, but not the value of reverse.

missing_first: bool = False

Affects the behavior for None values and empty strings.

If True, they should come before other values; if False, they should come after. This definition is based on the assumption that reverse=False; when reverse=True, the ultimate result will be the opposite.

reverse: bool = False

If True, sort descending.

class notesdir.models.FileQuerySortField(value)

Bases: enum.Enum

An enumeration.

CREATED = 'created'
FILENAME = 'filename'
PATH = 'path'
TAGS_COUNT = 'tags'
TITLE = 'title'
class notesdir.models.LinkInfo(referrer: str, href: str)

Bases: object

Represents a link from a file to some resource.

Not all links target local files, but those are the ones most important to notesdir. The referent() method can be used to determine what local file, if any, the href targets.

as_json() → dict

Returns a dict representing the instance, suitable for serializing as json.

href: str

The linked address.

Normally this is some sort of URI - it’s the address portion of a Markdown link, or the href or src attribute of an HTML tag, etc.

referent() → Optional[str]

Returns the resolved, absolute local path that this link refers to.

The path will be returned even if no file or folder actually exists at that location.

None will be returned if the href cannot be parsed or appears to be a non-file URI.

referrer: str

The file that contains the link. This should be a resolved, absolute path.

class notesdir.models.MoveCmd(path: str, dest: str, create_parents: bool = False, delete_empty_parents: bool = False)

Bases: notesdir.models.FileEditCmd

Represents a request to move a file or folder from one location to another.

This does not imply that any links should be rewritten; that is a higher-level operation, which is provided by notesdir.api.Notesdir.move().

create_parents: bool = False

If True, any nonexistent parent directories should be created.

delete_empty_parents: bool = False

If True, any parent directories that are empty after performing the move should be deleted.

dest: str

The new path and filename.

class notesdir.models.ReplaceHrefCmd(path: str, original: str, replacement: str)

Bases: notesdir.models.FileEditCmd

Represents a request to replace link addresses in a document.

All occurrences will be replaced, but only if they are exact matches.

original: str

The value to be replaced, generally copied from a LinkInfo href

replacement: str

The new link address.

class notesdir.models.SetCreatedCmd(path: str, value: Optional[datetime.datetime])

Bases: notesdir.models.FileEditCmd

Represents a request to change the creation date stored in a document’s metadata (not filesystem metadata).

value: Optional[datetime.datetime]

The new creation date, or None to delete it from the metadata.

class notesdir.models.SetTitleCmd(path: str, value: Optional[str])

Bases: notesdir.models.FileEditCmd

Represents a request to change a document’s title.

value: Optional[str]

The new title, or None to delete the title.

class notesdir.models.TemplateDirectives(dest: Optional[str] = None)

Bases: object

Passed by notesdir.api.Notesdir.new() when it is rendering one of a user’s templates.

It is used for passing data in and out of the template.

dest: Optional[str] = None

The path at which the new file should be created.

If this is set before rendering the template, it is the path the user suggested. But the template can change it, and the template’s suggestion will take precedence. If the path already exists, notesdir will adjust it further to get a unique path before creating the file.

notesdir.rearrange module

Helper functions for moving files and updating links between them.

Generally, you should use notesdir.api.Notesdir.move() or notesdir.api.Notesdir.replace_path_hrefs() instead of using anything in this module directly.

notesdir.rearrange.edits_for_path_replacement(referrer: str, hrefs: Set[str], replacement: str) → Iterator[notesdir.models.ReplaceHrefCmd]

Yields commands to replace a file’s links to a path with links to another path.

notesdir.rearrange.edits_for_raw_moves(renames: Dict[str, str]) → Iterator[notesdir.models.MoveCmd]

Yields commands that will rename a set of files/folders.

The keys of the dictionary are the paths to be renamed, and the values are what they should be renamed to. If a path appears as both a key and as a value, it will be moved to a temporary file as an intermediate step.

notesdir.rearrange.edits_for_rearrange(store: notesdir.repos.base.Repo, renames: Dict[str, str]) → Iterator[notesdir.models.FileEditCmd]

Yields commands that will rename files and update links accordingly.

The keys of the dictionary are the paths to be renamed, and the values are what they should be renamed to. (If a path appears as both a key and as a value, it will be moved to a temporary file as an intermediate step.)

The given store is used to search for files that link to any of the paths that are keys in the dictionary, so that ReplaceHrefEditCmd instances can be generated for them. The files that are being renamed will also be checked for outbound links, and ReplaceRef edits will be generated for those too.

Source paths may be directories; the directory as a whole will be moved, and links to/from all files/folders within it will be updated too.

notesdir.rearrange.find_available_name(dest: str, also_unavailable: Set[str], src: Optional[str] = None) → str

Given a desired destination path, adjusts it so that it will not overwrite any existing files.

dest should be the full desired path.

also_unavailable should be any paths that may be created between the time this method executes and the time the destination file is actually created.

If dest already exists or is in also_unavailable, a new path will be generated in the same directory, with a short UUID appended.

If you are moving an existing file, it is recommended to include the src parameter. Then, if the filename of dest and the filename of src are the same except that src includes a short UUID in the format generated by this function, this function will prefer keeping that UUID rather than generating a new one (but it will still prefer no UUID at all when possible).

notesdir.rearrange.href_path(src: str, dest: str) → str

Returns the path to use for a reference from file src to file dest.

This is a relative path to dest from the directory containing src.

For example, for src /foo/bar/baz.md and dest /foo/meh/blah.png, returns ../meh/blah.png.

src and dest are resolved before calculating the relative path.

notesdir.rearrange.path_as_href(path: str, into_url: Optional[urllib.parse.ParseResult] = None) → str

Returns the string to use for referring to the given path in a file.

This percent-encodes characters as necessary to make the path a valid URL. If into_url is provided, it copies every part of that URL except the path into the resulting URL.

Note that if into_url contains a scheme or netloc, the given path must be absolute.

Module contents

Helps manage notes stored as plain files in the filesystem.

If you installed via pip, run notesdir -h` to get help. Or, run ``python3 -m notesdir -h.

To use the Python API, look at notesdir.api.Notesdir