* Reapply "Make WS command backup/generate send events" (#131530)
This reverts commit 9b8316df3f.
* MVP implementation of Backup sync agents (#126122)
* init sync agent
* add syncing
* root import
* rename list to info and add sync state
* Add base backup class
* Revert unneded change
* adjust tests
* move to kitchen_sink
* split
* move
* Adjustments
* Adjustment
* update
* Tests
* Test unknown agent
* adjust
* Adjust for different test environments
* Change /info WS to contain a dictinary
* reorder
* Add websocket command to trigger sync from the supervisor
* cleanup
* Make mypy happier
---------
Co-authored-by: Erik <erik@montnemery.com>
* Make BackupSyncMetadata model a dataclass (#130555)
Make backup BackupSyncMetadata model a dataclass
* Rename backup sync agent to backup agent (#130575)
* Rename sync agent module to agent
* Rename BackupSyncAgent to BackupAgent
* Fix test typo
* Rename async_get_backup_sync_agents to async_get_backup_agents
* Rename and clean up remaining sync things
* Update kitchen sink
* Apply suggestions from code review
* Update test_manager.py
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Add additional options to WS command backup/generate (#130530)
* Add additional options to WS command backup/generate
* Improve test
* Improve test
* Align parameter names in backup/agents/* WS commands (#130590)
* Allow setting password for backups (#110630)
* Allow setting password for backups
* use is_hassio from helpers
* move it
* Fix getting psw
* Fix restoring with psw
* Address review comments
* Improve docstring
* Adjust kitchen sink
* Adjust
---------
Co-authored-by: Erik <erik@montnemery.com>
* Export relevant names from backup integration (#130596)
* Tweak backup agent interface (#130613)
* Tweak backup agent interface
* Adjust kitchen_sink
* Test kitchen sink backup (#130609)
* Test agents_list_backups
* Test agents_info
* Test agents_download
* Export Backup from manager
* Test agents_upload
* Update tests after rebase
* Use backup domain
* Remove WS command backup/upload (#130588)
* Remove WS command backup/upload
* Disable failing kitchen_sink test
* Make local backup a backup agent (#130623)
* Make local backup a backup agent
* Adjust
* Adjust
* Adjust
* Adjust tests
* Adjust
* Adjust
* Adjust docstring
* Adjust
* Protect members of CoreLocalBackupAgent
* Remove redundant check for file
* Make the backup.create service use the first local agent
* Add BackupAgent.async_get_backup
* Fix some TODOs
* Add support for downloading backup from a remote agent
* Fix restore
* Fix test
* Adjust kitchen_sink test
* Remove unused method BackupManager.async_get_backup_path
* Re-enable kitchen sink test
* Remove BaseBackupManager.async_upload_backup
* Support restore from remote agent
* Fix review comments
* Include backup agent error in response to WS command backup/info (#130884)
* Adjust code related to WS command backup/info (#130890)
* Include backup agent error in response to WS command backup/details (#130892)
* Remove LOCAL_AGENT_ID constant from backup manager (#130895)
* Add backup config storage (#130871)
* Add base for backup config
* Allow updating backup config
* Test loading backup config
* Add backup config update method
* Add temporary check for BackupAgent.async_remove_backup (#130893)
* Rename backup slug to backup_id (#130902)
* Improve backup websocket API tests (#130912)
* Improve backup websocket API tests
* Add missing snapshot
* Fix tests leaving files behind
* Improve backup manager backup creation tests (#130916)
* Remove class backup.backup.LocalBackup (#130919)
* Add agent delete backup (#130921)
* Add backup agent delete backup
* Remove agents delete websocket command
* Update docstring
Co-authored-by: Erik Montnemery <erik@montnemery.com>
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Disable core local backup agent in hassio (#130933)
* Rename remove backup to delete backup (#130940)
* Rename remove backup to delete backup
* Revert "backup/delete"
* Refactor BackupManager (#130947)
* Refactor BackupManager
* Adjust
* Adjust backup creation
* Copy in executor
* Fix BackupManager.async_get_backup (#130975)
* Fix typo in backup tests (#130978)
* Adjust backup NewBackup class (#130976)
* Remove class backup.BackupUploadMetadata (#130977)
Remove class backup.BackupMetadata
* Report backup size in bytes instead of MB (#131028)
Co-authored-by: Robert Resch <robert@resch.dev>
* Speed up CI for feature branch (#131030)
* Speed up CI for feature branch
* adjust
* fix
* fix
* fix
* fix
* Rename remove to delete in backup websocket type (#131023)
* Revert "Speed up CI for feature branch" (#131074)
Revert "Speed up CI for feature branch (#131030)"
This reverts commit 791280506d1859b1a722f5064d75bcbe48acc1c3.
* Rename class BaseBackup to AgentBackup (#131083)
* Rename class BaseBackup to AgentBackup
* Update tests
* Speed up CI for backup feature branch (#131079)
* Add backup platform to the hassio integration (#130991)
* Add backup platform to the hassio integration
* Add hassio to after_dependencies of backup
* Address review comments
* Remove redundant hassio parametrization of tests
* Add tests
* Address review comments
* Bump CI cache version
* Revert "Bump CI cache version"
This reverts commit 2ab4d2b1795c953ccfc9b17c47f9df3faac83749.
* Extend backup info class AgentBackup (#131110)
* Extend backup info class AgentBackup
* Update kitchen sink
* Update kitchen sink test
* Update kitchen sink test
* Exclude cloud and hassio from core files (#131117)
* Remove unnecessary **kwargs from backup API (#131124)
* Fix backup tests (#131128)
* Freeze backup dataclasses (#131122)
* Protect CoreLocalBackupAgent.load_backups (#131126)
* Use backup metadata v2 in core/container backups (#131125)
* Extend backup creation API (#131121)
* Extend backup creation API
* Add tests
* Fix merge
* Fix merge
* Return agent errors when deleting a backup (#131142)
* Return agent errors when deleting a backup
* Remove redundant calls to dict.keys()
* Add enum type for backup folder (#131158)
* Add method AgentBackup.from_dict (#131164)
* Remove WS command backup/agents/list_backups (#131163)
* Handle backup schedule (#131127)
* Add backup schedule handling
* Fix unrelated incorrect type annotation in test
* Clarify delay save
* Make the backup time compatible with the recorder nightly job
* Update create backup parameters
* Use typed dict for create backup parameters
* Simplify schedule state
* Group create backup parameters
* Move parameter
* Fix typo
* Use Folder model
* Handle deserialization of folders better
* Fail on attempt to include addons or folders in core backup (#131204)
* Fix AgentBackup test (#131201)
* Add options to WS command backup/restore (#131194)
* Add options to WS command backup/restore
* Add tests
* Fix test
* Teach core backup to restore only database or only settings (#131225)
* Exclude tmp_backups/*.tar from backups (#131243)
* Add WS command backup/subscribe_events (#131250)
* Clean up temporary directory after restoring backup (#131263)
* Improve hassio backup agent list (#131268)
* Include `last_automatic_backup` in reply to backup/info (#131293)
Include last_automatic_backup in reply to backup/info
* Handle backup delete after config (#131259)
* Handle delete after copies
* Handle delete after days
* Add some test examples
* Test config_delete_after_logic
* Test config_delete_after_copies_logic
* Test more delete after days
* Add debug logs
* Always delete the oldest backup first
* Never remove the last backup
* Clean up words
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Fix after cleaning words
* Use utcnow
* Remove duplicate guard
* Simplify sorting
* Delete backups even if there are agent errors on get backups
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* Rename backup delete after to backup retention (#131364)
* Rename backup delete after to backup retention
* Tweak
* Remove length limit on `agent_ids` when configuring backup (#132057)
Remove length limit on agent_ids when configuring backup
* Rename backup retention_config to retention (#132068)
* Modify backup agent API to be stream oriented (#132090)
* Modify backup agent API to be stream oriented
* Fix tests
* Adjust after code review
* Remove no longer needed pylint override
* Improve test coverage
* Change BackupAgent API to work with AsyncIterator objects
* Don't close files in the event loop
* Don't close files in the event loop
* Fix backup manager create backup log (#132174)
* Fix debug log level (#132186)
* Add cloud backup agent (#129621)
* Init cloud backup sync
* Add more metadata
* Fix typo
* Adjust to base changes
* Don't raise on list if more than one backup is available
* Adjust to base branch
* Fetch always and verify on download
* Update homeassistant/components/cloud/backup.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Adjust to base branch changes
* Not required anymore
* Workaround
* Fix blocking event loop
* Fix
* Add some tests
* some tests
* Add cloud backup delete functionality
* Enable check
* Fix ruff
* Use fixture
* Use iter_chunks instead
* Remove read
* Remove explicit export of read_backup
* Align with BackupAgent API changes
* Improve test coverage
* Improve error handling
* Adjust docstrings
* Catch aiohttp.ClientError bubbling up from hass_nabucasa
* Improve iteration
---------
Co-authored-by: Erik <erik@montnemery.com>
Co-authored-by: Robert Resch <robert@resch.dev>
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
Co-authored-by: Krisjanis Lejejs <krisjanis.lejejs@gmail.com>
* Extract file receiver from `BackupManager.async_receive_backup` to util (#132271)
* Extract file receiver from BackupManager.async_receive_backup to util
* Apply suggestions from code review
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Make sure backup directory exists (#132269)
* Make sure backup directory exists
* Hand off directory creation to executor
* Use mkdir's exist_ok feeature
* Organize BackupManager instance attributes (#132277)
* Don't store received backups in a TempDir (#132272)
* Don't store received backups in a TempDir
* Fix tests
* Make sure backup directory exists
* Address review comments
* Fix tests
* Rewrite backup manager state handling (#132375)
* Rewrite backup manager state handling
* Address review comments
* Modify backup reader/writer API to be stream oriented (#132464)
* Internalize backup tasks (#132482)
* Internalize backup tasks
* Update test after rebase
* Handle backup error during automatic backup (#132511)
* Improve backup manager state logging (#132549)
* Fix backup manager state when restore completes (#132548)
* Remove WS command backup/agents/download (#132664)
* Add WS command backup/generate_with_stored_settings (#132671)
* Add WS command backup/generate_with_stored_settings
* Register the new command, add tests
* Refactor local agent backup tests (#132683)
* Refactor test_load_backups
* Refactor test loading agents
* Refactor test_delete_backup
* Refactor test_upload
* Clean up duplicate tests
* Refactor backup manager receive tests (#132701)
* Refactor backup manager receive tests
* Clean up
* Refactor pre and post platform tests (#132708)
* Refactor backup pre platform test
* Refactor backup post platform test
* Bump aiohasupervisor to version 0.2.2b0 (#132704)
* Bump aiohasupervisor to version 0.2.2b0
* Adjust tests
* Publish event when manager is idle after creating backup (#132724)
* Handle busy backup manager when uploading backup (#132736)
* Adjust hassio backup agent to supervisor changes (#132732)
* Adjust hassio backup agent to supervisor changes
* Fix typo
* Refactor test for create backup with wrong parameters (#132763)
* Refactor test not loading bad backup platforms (#132769)
* Improve receive backup coverage (#132758)
* Refactor initiate backup test (#132829)
* Rename Backup to ManagerBackup (#132841)
* Refactor backup config (#132845)
* Refactor backup config
* Remove unnecessary condition
* Adjust tests
* Improve initiate backup test (#132858)
* Store the time of automatic backup attempts (#132860)
* Store the time of automatic backup attempts
* Address review comments
* Update test
* Update cloud test
* Save agent failures when creating backups (#132850)
* Save agent failures when creating backups
* Update tests
* Store KnownBackups
* Add test
* Only clear known_backups on no error, add tests
* Address review comments
* Store known backups as a list
* Update tests
* Track all backups created with backup strategy settings (#132916)
* Track all backups created with saved settings
* Rename
* Add explicit call to save the store
* Don't register service backup.create in HassOS installations (#132932)
* Revert changes to action service backup.create (#132938)
* Fix logic for cleaning up temporary backup file (#132934)
* Fix logic for cleaning up temporary backup file
* Reduce scope of patch
* Fix with_strategy_settings info not sent over websocket (#132939)
* Fix with_strategy_settings info not sent over websocket
* Fix kitchen sink tests
* Fix cloud and hassio tests
* Revert backup ci changes (#132955)
Revert changes speeding up CI
* Fix revert of CI changes (#132960)
---------
Co-authored-by: Joakim Sørensen <joasoe@gmail.com>
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
Co-authored-by: Robert Resch <robert@resch.dev>
Co-authored-by: Paul Bottein <paul.bottein@gmail.com>
Co-authored-by: Krisjanis Lejejs <krisjanis.lejejs@gmail.com>
182 lines
5.9 KiB
Python
182 lines
5.9 KiB
Python
"""Home Assistant module to handle restoring backups."""
|
|
|
|
from __future__ import annotations
|
|
|
|
from collections.abc import Iterable
|
|
from dataclasses import dataclass
|
|
import hashlib
|
|
import json
|
|
import logging
|
|
from pathlib import Path
|
|
import shutil
|
|
import sys
|
|
from tempfile import TemporaryDirectory
|
|
|
|
from awesomeversion import AwesomeVersion
|
|
import securetar
|
|
|
|
from .const import __version__ as HA_VERSION
|
|
|
|
RESTORE_BACKUP_FILE = ".HA_RESTORE"
|
|
KEEP_BACKUPS = ("backups",)
|
|
KEEP_DATABASE = (
|
|
"home-assistant_v2.db",
|
|
"home-assistant_v2.db-wal",
|
|
)
|
|
|
|
|
|
_LOGGER = logging.getLogger(__name__)
|
|
|
|
|
|
@dataclass
|
|
class RestoreBackupFileContent:
|
|
"""Definition for restore backup file content."""
|
|
|
|
backup_file_path: Path
|
|
password: str | None
|
|
remove_after_restore: bool
|
|
restore_database: bool
|
|
restore_homeassistant: bool
|
|
|
|
|
|
def password_to_key(password: str) -> bytes:
|
|
"""Generate a AES Key from password.
|
|
|
|
Matches the implementation in supervisor.backups.utils.password_to_key.
|
|
"""
|
|
key: bytes = password.encode()
|
|
for _ in range(100):
|
|
key = hashlib.sha256(key).digest()
|
|
return key[:16]
|
|
|
|
|
|
def restore_backup_file_content(config_dir: Path) -> RestoreBackupFileContent | None:
|
|
"""Return the contents of the restore backup file."""
|
|
instruction_path = config_dir.joinpath(RESTORE_BACKUP_FILE)
|
|
try:
|
|
instruction_content = json.loads(instruction_path.read_text(encoding="utf-8"))
|
|
return RestoreBackupFileContent(
|
|
backup_file_path=Path(instruction_content["path"]),
|
|
password=instruction_content["password"],
|
|
remove_after_restore=instruction_content["remove_after_restore"],
|
|
restore_database=instruction_content["restore_database"],
|
|
restore_homeassistant=instruction_content["restore_homeassistant"],
|
|
)
|
|
except (FileNotFoundError, KeyError, json.JSONDecodeError):
|
|
return None
|
|
|
|
|
|
def _clear_configuration_directory(config_dir: Path, keep: Iterable[str]) -> None:
|
|
"""Delete all files and directories in the config directory except entries in the keep list."""
|
|
keep_paths = [config_dir.joinpath(path) for path in keep]
|
|
entries_to_remove = sorted(
|
|
entry for entry in config_dir.iterdir() if entry not in keep_paths
|
|
)
|
|
|
|
for entry in entries_to_remove:
|
|
entrypath = config_dir.joinpath(entry)
|
|
|
|
if entrypath.is_file():
|
|
entrypath.unlink()
|
|
elif entrypath.is_dir():
|
|
shutil.rmtree(entrypath)
|
|
|
|
|
|
def _extract_backup(
|
|
config_dir: Path,
|
|
restore_content: RestoreBackupFileContent,
|
|
) -> None:
|
|
"""Extract the backup file to the config directory."""
|
|
with (
|
|
TemporaryDirectory() as tempdir,
|
|
securetar.SecureTarFile(
|
|
restore_content.backup_file_path,
|
|
gzip=False,
|
|
mode="r",
|
|
) as ostf,
|
|
):
|
|
ostf.extractall(
|
|
path=Path(tempdir, "extracted"),
|
|
members=securetar.secure_path(ostf),
|
|
filter="fully_trusted",
|
|
)
|
|
backup_meta_file = Path(tempdir, "extracted", "backup.json")
|
|
backup_meta = json.loads(backup_meta_file.read_text(encoding="utf8"))
|
|
|
|
if (
|
|
backup_meta_version := AwesomeVersion(
|
|
backup_meta["homeassistant"]["version"]
|
|
)
|
|
) > HA_VERSION:
|
|
raise ValueError(
|
|
f"You need at least Home Assistant version {backup_meta_version} to restore this backup"
|
|
)
|
|
|
|
with securetar.SecureTarFile(
|
|
Path(
|
|
tempdir,
|
|
"extracted",
|
|
f"homeassistant.tar{'.gz' if backup_meta["compressed"] else ''}",
|
|
),
|
|
gzip=backup_meta["compressed"],
|
|
key=password_to_key(restore_content.password)
|
|
if restore_content.password is not None
|
|
else None,
|
|
mode="r",
|
|
) as istf:
|
|
istf.extractall(
|
|
path=Path(tempdir, "homeassistant"),
|
|
members=securetar.secure_path(istf),
|
|
filter="fully_trusted",
|
|
)
|
|
if restore_content.restore_homeassistant:
|
|
keep = list(KEEP_BACKUPS)
|
|
if not restore_content.restore_database:
|
|
keep.extend(KEEP_DATABASE)
|
|
_clear_configuration_directory(config_dir, keep)
|
|
shutil.copytree(
|
|
Path(tempdir, "homeassistant", "data"),
|
|
config_dir,
|
|
dirs_exist_ok=True,
|
|
ignore=shutil.ignore_patterns(*(keep)),
|
|
)
|
|
elif restore_content.restore_database:
|
|
for entry in KEEP_DATABASE:
|
|
entrypath = config_dir / entry
|
|
|
|
if entrypath.is_file():
|
|
entrypath.unlink()
|
|
elif entrypath.is_dir():
|
|
shutil.rmtree(entrypath)
|
|
|
|
for entry in KEEP_DATABASE:
|
|
shutil.copy(
|
|
Path(tempdir, "homeassistant", "data", entry),
|
|
config_dir,
|
|
)
|
|
|
|
|
|
def restore_backup(config_dir_path: str) -> bool:
|
|
"""Restore the backup file if any.
|
|
|
|
Returns True if a restore backup file was found and restored, False otherwise.
|
|
"""
|
|
config_dir = Path(config_dir_path)
|
|
if not (restore_content := restore_backup_file_content(config_dir)):
|
|
return False
|
|
|
|
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
|
|
backup_file_path = restore_content.backup_file_path
|
|
_LOGGER.info("Restoring %s", backup_file_path)
|
|
try:
|
|
_extract_backup(
|
|
config_dir=config_dir,
|
|
restore_content=restore_content,
|
|
)
|
|
except FileNotFoundError as err:
|
|
raise ValueError(f"Backup file {backup_file_path} does not exist") from err
|
|
if restore_content.remove_after_restore:
|
|
backup_file_path.unlink(missing_ok=True)
|
|
_LOGGER.info("Restore complete, restarting")
|
|
return True
|