ffiec_data_connect package

Submodules

ffiec_data_connect.constants module

Constant values utilized for data collection and other purposes.

This module contains constant values that are unlikely to change, but need to be referenced by other modules.

class WebserviceConstants[source]

Bases: object

The URL endpoint for the FFIEC SOAP-based webservice.

base_url = 'https://cdr.ffiec.gov/Public/PWS/WebServices/RetrievalService.asmx?WSDL'

ffiec_data_connect.credentials module

Methods to utilize for inputting credentials for the FFIEC data connection.

This module provides secure methods for inputting credentials for the FFIEC webservice data connection.

Credentials may be input via environment variables, or passing them as arguments into the class structure. Wherever possible, the credentials should not be stored in source code.

class CredentialType(*values)[source]

Bases: Enum

Enumerated values that represent the methods through which credentials are provided to the FFIEC webservice via the package.

Parameters:: Enum (integer) – Integer that represents the credential input method

NO_CREDENTIALS = 0

SET_FROM_ENV = 2

SET_FROM_INIT = 1

class OAuth2Credentials(username: str, bearer_token: str, token_expires: datetime | None = None)[source]

Bases: object

OAuth2-based credentials for REST API access - Phase 0 Implementation.

This credential type supports the new FFIEC REST API that uses OAuth2 Bearer tokens for authentication instead of username/password pairs.

Key Features: - OAuth2 Bearer token authentication (90-day lifecycle) - Token expiration tracking and validation - Immutable after initialization for security - Compatible with automatic protocol selection

Parameters:

username – FFIEC username (for UserID header)
bearer_token – OAuth2 bearer token (90-day lifecycle)
token_expires – Optional token expiration datetime

Example:

# Create OAuth2 credentials for REST API
creds = OAuth2Credentials(
    username="your_ffiec_username",
    bearer_token="your_90_day_bearer_token",
    token_expires=datetime(2024, 3, 15)  # Optional
)

# Use with existing methods (automatic REST API selection)
periods = collect_reporting_periods(session, creds)

property bearer_token: str

Get the OAuth2 bearer token.

Returns:: Bearer token for authentication

get_auth_headers() → Dict[str, str][source]

Get authentication headers for REST API requests.

Returns:: Dictionary containing required authentication headers

property is_expired: bool

Check if token is expired or expires within 24 hours.

Returns:: True if token is expired or expires soon, False otherwise

test_credentials(session: Session = None) → bool[source]

Test OAuth2 credentials against REST API (placeholder).

Note: This will be implemented when REST adapter is available. For now, validates token format and expiration.

Parameters:: session – Optional requests session (for future compatibility)
Returns:: True if credentials appear valid, False otherwise

property token_expires: datetime | None

Get token expiration datetime.

Returns:: Token expiration datetime or None if not set

property username: str

Get the FFIEC username.

Returns:: Username for UserID header

class WebserviceCredentials(username: str | None = None, password: str | None = None)[source]

Bases: object

The WebserviceCredentials class. This class is used to store the credentials for the FFIEC webservice.

Parameters:

username (str, optional) – FFIEC Webservice username. Optional: If not provided, the credentials will be set from the environment variable FFIEC_USERNAME
password (str, optional) – FFIEC Webservice password. Optional: If not provided, the credentials will be set from the environment variable FFIEC_PASSWORD

Returns:

An instance of the WebserviceCredentials class.

Return type:

WebserviceCredentials

property password: str

Returns the password from the WebserviceCredentials instance.

Returns:: the password stored in the WebserviceCredentials instance
Return type:: str

test_credentials(session: Session) → bool[source]

Test the credentials with the FFIEC Webservice to determine if they are valid and accepted.

Note: The session argument can be generated directly from requests, or

using the helper class FFIECConnection

Parameters:

session (requests.Session) – the connection to test the credentials

Returns:

True if the credentials are valid, False otherwise

Return type:

bool

Raises:

ValueError – if the credentials are not set
Exception – Other unspecified errors

property username: str

Returns the username from the WebserviceCredentials instance.

Returns:: the username stored in the WebserviceCredentials instance
Return type:: str

ffiec_data_connect.datahelpers module

Internal functions to assist with processing results from the Webservice

ffiec_data_connect.exceptions module

Descriptive exceptions for the ffiec_data_connect package

This module provides custom exceptions with detailed error messages to improve debugging and user experience.

exception ConnectionError(message: str, url: str | None = None, status_code: int | None = None)[source]

Bases: FFIECError

Raised when connection to FFIEC webservice fails

exception CredentialError(message: str, credential_source: str | None = None)[source]

Bases: FFIECError

Raised when there are issues with credentials

exception FFIECError(message: str, details: Dict[str, Any] | None = None)[source]

Bases: Exception

Base exception for all FFIEC Data Connect errors

exception NoDataError(rssd_id: str | None = None, reporting_period: str | None = None)[source]

Bases: FFIECError

Raised when no data is returned from the FFIEC webservice

exception RateLimitError(retry_after: int | None = None)[source]

Bases: FFIECError

Raised when API rate limit is exceeded

exception SessionError(message: str, session_state: str | None = None)[source]

Bases: FFIECError

Raised when there are session management issues

exception ValidationError(field: str, value: Any, expected: str)[source]

Bases: FFIECError

Raised when input validation fails

exception XMLParsingError(message: str, xml_snippet: str | None = None)[source]

Bases: FFIECError

Raised when XML/XBRL parsing fails

raise_exception(exception_class: Type[Exception], legacy_message: str, *args: Any, **kwargs: Any) → None[source]

Raise an exception with legacy compatibility support.

If legacy mode is enabled, raises ValueError with the legacy message. Otherwise, raises the specific exception class.

Parameters:

exception_class – The specific exception class to raise
legacy_message – Message to use for ValueError in legacy mode
*args – Arguments for the specific exception
**kwargs – Keyword arguments for the specific exception

ffiec_data_connect.ffiec_connection module

Wrapper to establish requests.Session and set proxy server values if needed

An instance of this class may be substituted for an instance requests.Session class.

class FFIECConnection[source]

Bases: object

Thread-safe FFIECConnection with proper resource management.

This class provides a thread-safe connection to the FFIEC webservice with optional proxy support and automatic resource cleanup.

classmethod cleanup_all() → None[source]

Class method to cleanup all registered instances.

This can be useful for cleanup in testing or shutdown scenarios.

close() → None[source]

Close the session and free resources.

This method should be called when the connection is no longer needed to ensure proper cleanup of network resources.

property proxy_host: str | None

Returns the hostname of the proxy server

Returns:: The hostname of the proxy server
Return type:: str

property proxy_password: str | None

Get the optional proxy password

Returns:: the password of the proxy server
Return type:: str

property proxy_port: int | None

Get the optional proxy port

Returns:: the proxy port
Return type:: int

property proxy_protocol: ProxyProtocol | None

Returns the protocol of the proxy server

Returns:: the protocol of the proxy server
Return type:: int

property proxy_user_name: str | None

Get the optional proxy username

Returns:: the proxy username
Return type:: str

property session: Session

Returns the requests.Session object (lazy-loaded and thread-safe).

Note that this property may be utilized for methods in the methods module.
The session is created on first access and reused thereafter.
Thread-safe: Multiple threads can safely access this property.

Returns:: the requests.Session object
Return type:: requests.Session

test_connection(url: str = 'https://google.com') → bool[source]

Tests a connection, using the proxy server if one is not set

Note: This method tests if a web page on the public internet (not the FFIEC webservice) is accessible using this library, through the proxy server.

We do not yet test access to the Webservice, because a user name and password is needed to access the Webservice, which is outside the scope of this module.

An alternative web site may be selected in lieu of google.com, using the url parameter.

property use_proxy: bool

Get the optional proxy flag

Returns:: the proxy flag (True if proxy is used, False if not)
Return type:: bool

class ProxyProtocol(*values)[source]

Bases: Enum

Enumerated values that represent the proxy protocol options

Parameters:: Enum (ProxyProtocol) – Enumerated value for HTTP or HTTPS

HTTP = 0

HTTPS = 1

ffiec_data_connect.methods module

Methods that wrap the FFIEC Webservice API

The methods contained in this module are utilized to call and collect data from the FFIEC Webservice API.

collect_data(session: FFIECConnection | Session | None, creds: WebserviceCredentials | OAuth2Credentials, reporting_period: str | datetime, rssd_id: str, series: str, output_type: str = 'list', date_output_format: str = 'string_original', force_null_types: str | None = None) → Any[source]

Return time series data from the FFIEC webservice for a given reporting period and RSSD ID

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Translates the input reporting period to a FFIEC-formatted date Transforms the output to a pandas dataframe if output_type is ‘pandas’, otherwise returns a list

Valid arguments for the ``reporting_period` argument:

mm/dd/yyyy
yyyy-mm-dd
yyyymmdd
a python datetime object
For the above types, the date msut be the last day in the quarter (e.g. March 31, June 30, September 30, or December 31)
#Qyyyy, where # is the quarter number and yyyy is the year.

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
reporting_period (str or datetime) – Reporting period.
rssd_id (str) – The RSSD ID of the entity for which you want to retrieve data.
series (str) – call or ubpr
output_type (str) – list, pandas, or polars
date_output_format (str) – string_original, string_yyyymmdd, or python_format
force_null_types (str, optional) – Override null value handling. Options: - None (default): Automatic based on API (SOAP uses numpy, REST uses pandas) - “numpy”: Force np.nan for null values (original behavior) - “pandas”: Force pd.NA for null values (better integer handling)

Returns:

Returns data in the specified format

Return type:

list, pandas DataFrame, or polars DataFrame

Retrieves data from FFIEC webservice.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Retrieves the Financial Institutions in a Panel of Reporters for a given reporting period. Note that this function only reports on Call Report filings, not UBPR filings.

Valid arguments for the ``reporting_period` argument:

mm/dd/yyyy
yyyy-mm-dd
yyyymmdd
a python datetime object
For the above types, the date must be the last day in the quarter (e.g. March 31, June 30, September 30, or December 31)
#Qyyyy, where # is the quarter number and yyyy is the year.

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
reporting_period (str or datetime) – The reporting period to use for the request.

Returns:

List of dicts or pandas DataFrame containing the following fields: - “rssd”/”id_rssd”: Institution RSSD ID (both field names provided for compatibility) - “fdic_cert_number”: FDIC certificate number - “occ_chart_number”: OCC charter number - “ots_dock_number”: OTS docket number - “primary_aba_rout_number”: Primary ABA routing number - “name”: Institution name - “state”: State - “city”: City - “address”: Street address - “filing_type”: Filing type - “has_filed_for_reporting_period”: Whether institution has filed for the period

NOTE: Property names were inconsistent in earlier code, so both ‘rssd’ and ‘id_rssd’ are provided with identical data to reduce need to refactor existing user code.

Return type:

list or pd.DataFrame

Retrieves data from FFIEC webservice.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Retrieves the ID RSSDs of the reporters who have filed after a given date for a given reporting period. Note that this function only reports on Call Report filings, not UBPR filings.

Valid arguments for the ``since_date` argument:

mm/dd/yyyy

yyyy-mm-dd

yyyymmdd

a python datetime object

Valid arguments for the ``reporting_period` argument:

all of the above, as long as the date is the last day in the quarter (e.g. March 31, June 30, September 30, or December 31)

#Qyyyy, where # is the quarter number and yyyy is the year.

Args:
session (FFIECConnection or requests.Session): The requests session object to use for the request. creds (WebserviceCredentials): The credentials to use for the request. since_date (str or datetime): The date to use for the request. May be in the format of ‘YYYY-MM-DD’, ‘YYYYMMDD’, ‘MM/DD/YYYY’, or a python datetime object. output_type (str, optional): “list” or “pandas”. Defaults to “list”.

Returns:
any: Returns either a list of dicts or a pandas Series comprising the ID RSSDs of the reporters who have filed after a given date for a given reporting period.

collect_filers_submission_date_time(session: FFIECConnection | Session | None, creds: WebserviceCredentials | OAuth2Credentials, since_date: str | datetime, reporting_period: str | datetime, output_type: str = 'list', date_output_format: str = 'string_original') → List[Any] | DataFrame[source]

Retrieves data from FFIEC webservice.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Retrieves the ID RSSDs and DateTime of the reporters who have filed after a given date for a given reporting period. Note that this function only reports on Call Report filings, not UBPR filings.

Note on date_output_format:

string_original is the default output format, and is the format that is used by the FFIEC webservice: mm/dd/yyyy

string_yyyymmdd is the date in yyyymmdd format

python_format is the date in python datetime format

Args:
session (ffiec_connection.FFIECConnection or requests.Session): The requests session object to use for the request. creds (WebserviceCredentials or requests.Session): The credentials to use for the request. since_date (str or datetime): The date to use for the request. May be in the format of ‘YYYY-MM-DD’, ‘YYYYMMDD’, ‘MM/DD/YYYY’, or a python datetime object. reporting_period (str or datetime): The reporting period to use for the request (e.g. “2020-03-21”). Note that the date must be in the format of “YYYY-MM-DD”, “YYYYMMDD”, “MM/DD/YYYY”, #QYYYY or a python datetime object, with the month and date set to March 31, June 30, September 30, or December 31. output_type (str, optional): “list” or “pandas”. Defaults to “list”. date_output_format (str, optional): string_original or python_datetime. Defaults to “string_original”.

Returns:
any: List of dicts or pandas DataFrame containing the following fields: - “rssd”/”id_rssd”: Institution RSSD ID (both field names provided for compatibility) - “datetime”: Submission date and time in Washington DC timezone

NOTE: Property names were inconsistent in earlier code, so both ‘rssd’ and ‘id_rssd’ are provided with identical data to reduce need to refactor existing user code.

collect_reporting_periods(session: FFIECConnection | Session | None, creds: WebserviceCredentials | OAuth2Credentials, series: str = 'call', output_type: str = 'list', date_output_format: str = 'string_original') → List[str] | List[datetime] | Series[source]

Returns list of reporting periods available for access via the FFIEC webservice

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Note on date_output_format:

string_original is the default output format, and is the format that is used by the FFIEC webservice: mm/dd/yyyy
string_yyyymmdd is the date in yyyymmdd format
python_format is the date in python datetime format

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
series (str, optional) – call or ubpr
output_type (str) – list or pandas
date_output_format – string_original, string_yyyymmdd, or python_format

Returns:

Returns a list of reporting periods from the FFIEC Webservice in ascending chronological order (oldest first)

Return type:

list or Pandas series

Retrieves UBPR XBRL facsimile data for a specific institution.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
reporting_period – Reporting period date
rssd_id – Institution RSSD ID
output_type – Output format (“list”, “pandas”, “polars”, or “bytes”)
force_null_types (str, optional) – Override null value handling. Options: - None (default): Automatic based on API (REST uses pandas) - “numpy”: Force np.nan for null values - “pandas”: Force pd.NA for null values

Returns:

UBPR XBRL data

Return type:

bytes, list, or pd.DataFrame

collect_ubpr_reporting_periods(session: FFIECConnection | Session | None, creds: WebserviceCredentials | OAuth2Credentials, output_type: str = 'list', date_output_format: str = 'string_original') → List[Any] | DataFrame[source]

Retrieves UBPR reporting periods from FFIEC API.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
output_type – Output format (“list”, “pandas”, or “polars”)
date_output_format – Date format for output

Returns:

List of UBPR reporting periods in ascending chronological order (oldest first)

Return type:

list or pd.DataFrame

ffiec_data_connect.xbrl_processor module

Internal functions used to process XBRL data received from the FFIEC Webservice

This module provides secure XML/XBRL processing with XXE attack prevention.

Module contents

FFIEC Data Connect - Python wrapper for FFIEC webservice API.

This package provides secure, thread-safe access to FFIEC financial data with support for both REST and SOAP APIs.

class AsyncCompatibleClient(credentials: WebserviceCredentials | OAuth2Credentials, max_concurrent: int = 5, rate_limit: float | None = 10, executor: ThreadPoolExecutor | None = None)[source]

Bases: object

Client that supports both sync and async usage patterns.

This client provides: - Backward compatible synchronous methods - Parallel processing with thread pools - Async/await support for integration with async frameworks - Rate limiting to respect API limits - Thread-safe operation

close() → None[source]: Close all connections and cleanup resources.

async collect_batch_async(reporting_period: str, rssd_ids: List[str], series: str = 'call', output_type: str = 'list', date_output_format: str = 'string_original', progress_callback: Callable[[str, Any], None] | None = None) → Dict[str, List[Dict[str, Any]] | Dict[str, Any]][source]

Collect data for multiple banks with rate limiting and progress tracking.

Parameters:

reporting_period – Reporting period
rssd_ids – List of RSSD IDs
series – Data series
output_type – Output format
date_output_format – Date format
progress_callback – Optional async callback for progress

Returns:

Dictionary mapping RSSD IDs to their data

collect_data(reporting_period: str, rssd_id: str, series: str = 'call', output_type: str = 'list', date_output_format: str = 'string_original') → List[Dict[str, Any]] | Any[source]

Standard synchronous method - backward compatible.

Parameters:

reporting_period – Reporting period (e.g., “2020-03-31” or “1Q2020”)
rssd_id – RSSD ID of the institution
series – Data series (“call” or “ubpr”)
output_type – Output format (“list”, “pandas”, or “polars”)
date_output_format – Date format in output

Returns:

Collected data in requested format

async collect_data_async(reporting_period: str, rssd_id: str, series: str = 'call', output_type: str = 'list', date_output_format: str = 'string_original') → List[Dict[str, Any]] | Any[source]

Async version - runs sync code in thread pool.

Parameters:

reporting_period – Reporting period
rssd_id – RSSD ID of the institution
series – Data series
output_type – Output format
date_output_format – Date format

Returns:

Collected data in requested format

collect_data_parallel(reporting_period: str, rssd_ids: List[str], series: str = 'call', output_type: str = 'list', date_output_format: str = 'string_original', progress_callback: Callable[[str, Any], None] | None = None) → Dict[str, List[Dict[str, Any]] | Dict[str, Any]][source]

Collect data for multiple banks in parallel (sync interface).

Parameters:

reporting_period – Reporting period
rssd_ids – List of RSSD IDs
series – Data series
output_type – Output format
date_output_format – Date format
progress_callback – Optional callback for progress updates

Returns:

Dictionary mapping RSSD IDs to their data or error info

collect_reporting_periods(series: str = 'call', output_type: str = 'list', date_output_format: str = 'string_original') → List[str] | Any[source]

Get available reporting periods - backward compatible.

Parameters:

series – Data series (“call” or “ubpr”)
output_type – Output format (“list”, “pandas”, or “polars”)
date_output_format – Date format in output

Returns:

Available reporting periods in ascending chronological order (oldest first)

collect_time_series(rssd_id: str, reporting_periods: List[str], series: str = 'call', output_type: str = 'list', date_output_format: str = 'string_original') → Dict[str, List[Dict[str, Any]] | Dict[str, Any]][source]

Collect multiple periods for one bank in parallel (sync interface).

Parameters:

rssd_id – RSSD ID of the institution
reporting_periods – List of reporting periods
series – Data series
output_type – Output format
date_output_format – Date format

Returns:

Dictionary mapping periods to their data

async collect_time_series_async(rssd_id: str, reporting_periods: List[str], series: str = 'call', output_type: str = 'list', date_output_format: str = 'string_original') → Dict[str, List[Dict[str, Any]] | Dict[str, Any]][source]

Collect multiple periods for one bank in parallel (async).

Parameters:

rssd_id – RSSD ID of the institution
reporting_periods – List of reporting periods
series – Data series
output_type – Output format
date_output_format – Date format

Returns:

Dictionary mapping periods to their data

exception ConnectionError(message: str, url: str | None = None, status_code: int | None = None)[source]

Bases: FFIECError

Raised when connection to FFIEC webservice fails

exception CredentialError(message: str, credential_source: str | None = None)[source]

Bases: FFIECError

Raised when there are issues with credentials

class CredentialType(*values)[source]

Bases: Enum

Enumerated values that represent the methods through which credentials are provided to the FFIEC webservice via the package.

Parameters:: Enum (integer) – Integer that represents the credential input method

NO_CREDENTIALS = 0

SET_FROM_ENV = 2

SET_FROM_INIT = 1

class DataNormalizer[source]

Bases: object

Ensures 100% backward compatibility by normalizing REST responses to SOAP format.

This class prevents data regressions by applying type coercions and format transformations to REST API responses, making them identical to SOAP responses.

TYPE_COERCIONS: Dict[str, Dict[str, Any]] = {'RetrieveFacsimile': {'_preserve_binary': True}, 'RetrieveFilersSinceDate': {'_array_items': <function DataNormalizer.<lambda>>}, 'RetrieveFilersSubmissionDateTime': {'ID_RSSD': <function DataNormalizer.<lambda>>, 'SubmissionDateTime': <function DataNormalizer.<lambda>>}, 'RetrievePanelOfReporters': {'FDICCertNumber': <function DataNormalizer.<lambda>>, 'HasFiledForReportingPeriod': <function DataNormalizer.<lambda>>, 'ID_RSSD': <function DataNormalizer.<lambda>>, 'InstitutionName': <function DataNormalizer.<lambda>>, 'MailingCity': <function DataNormalizer.<lambda>>, 'MailingState': <function DataNormalizer.<lambda>>, 'MailingStreetAddress': <function DataNormalizer.<lambda>>, 'MailingZIP': <function DataNormalizer.<lambda>>, 'OCCChartNumber': <function DataNormalizer.<lambda>>, 'OTSDockNumber': <function DataNormalizer.<lambda>>, 'PhysicalCity': <function DataNormalizer.<lambda>>, 'PhysicalState': <function DataNormalizer.<lambda>>, 'PhysicalStreetAddress': <function DataNormalizer.<lambda>>, 'PrimaryABARoutNumber': <function DataNormalizer.<lambda>>, 'ZIP': <function DataNormalizer.<lambda>>}, 'RetrieveReportingPeriods': {'_array_items': <function DataNormalizer.<lambda>>}, 'RetrieveUBPRReportingPeriods': {'_array_items': <function DataNormalizer.<lambda>>}, 'RetrieveUBPRXBRLFacsimile': {'_preserve_binary': True}}

VALIDATION_RULES = {'FDICCertNumber': {'description': 'FDIC certificate number as string', 'pattern': '^\\d*$'}, 'ID_RSSD': {'description': 'RSSD ID as string of digits', 'pattern': '^\\d+$'}, 'ZIP': {'description': '5-digit ZIP code with leading zeros preserved', 'pattern': '^\\d{5}$'}}

static get_normalization_stats(data_before: Any, data_after: Any, endpoint: str) → Dict[str, Any][source]

Generate statistics about normalization transformations applied.

Useful for monitoring and debugging normalization effectiveness.

static normalize_for_validation(data: Any, endpoint: str, protocol: str = 'REST') → Tuple[Any, Dict[str, Any]][source]

Normalize response data and return both normalized data and statistics.

This method is designed to work with Pydantic validation by providing both the normalized data and metadata about transformations applied.

Parameters:

data – Raw response data from API
endpoint – API endpoint name
protocol – Source protocol (“REST” or “SOAP”)

Returns:

Tuple of (normalized_data, normalization_stats)

static normalize_response(data: Any, endpoint: str, protocol: str = 'REST') → Any[source]

Normalize REST response to match SOAP format exactly.

This is the main entry point for data normalization. It ensures that REST API responses are transformed to be identical to SOAP responses, preventing any user-visible changes during protocol migration.

Parameters:

data – Raw response data from API
endpoint – API endpoint name (e.g., “RetrievePanelOfReporters”)
protocol – Source protocol (“REST” or “SOAP”)

Returns:

Normalized data matching SOAP format exactly

static validate_pydantic_compatibility(data: Any, endpoint: str) → Dict[str, Any][source]

Check if normalized data is compatible with expected Pydantic models.

This performs additional validation beyond basic normalization to ensure data will pass Pydantic validation without issues.

Parameters:

data – Normalized data
endpoint – API endpoint name

Returns:

Dictionary with validation results and recommendations

class FFIECConnection[source]

Bases: object

Thread-safe FFIECConnection with proper resource management.

This class provides a thread-safe connection to the FFIEC webservice with optional proxy support and automatic resource cleanup.

classmethod cleanup_all() → None[source]

Class method to cleanup all registered instances.

This can be useful for cleanup in testing or shutdown scenarios.

close() → None[source]

Close the session and free resources.

This method should be called when the connection is no longer needed to ensure proper cleanup of network resources.

property proxy_host: str | None

Returns the hostname of the proxy server

Returns:: The hostname of the proxy server
Return type:: str

property proxy_password: str | None

Get the optional proxy password

Returns:: the password of the proxy server
Return type:: str

property proxy_port: int | None

Get the optional proxy port

Returns:: the proxy port
Return type:: int

property proxy_protocol: ProxyProtocol | None

Returns the protocol of the proxy server

Returns:: the protocol of the proxy server
Return type:: int

property proxy_user_name: str | None

Get the optional proxy username

Returns:: the proxy username
Return type:: str

property session: Session

Returns the requests.Session object (lazy-loaded and thread-safe).

Note that this property may be utilized for methods in the methods module.
The session is created on first access and reused thereafter.
Thread-safe: Multiple threads can safely access this property.

Returns:: the requests.Session object
Return type:: requests.Session

test_connection(url: str = 'https://google.com') → bool[source]

Tests a connection, using the proxy server if one is not set

Note: This method tests if a web page on the public internet (not the FFIEC webservice) is accessible using this library, through the proxy server.

We do not yet test access to the Webservice, because a user name and password is needed to access the Webservice, which is outside the scope of this module.

An alternative web site may be selected in lieu of google.com, using the url parameter.

property use_proxy: bool

Get the optional proxy flag

Returns:: the proxy flag (True if proxy is used, False if not)
Return type:: bool

exception FFIECError(message: str, details: Dict[str, Any] | None = None)[source]

Bases: Exception

Base exception for all FFIEC Data Connect errors

exception NoDataError(rssd_id: str | None = None, reporting_period: str | None = None)[source]

Bases: FFIECError

Raised when no data is returned from the FFIEC webservice

class OAuth2Credentials(username: str, bearer_token: str, token_expires: datetime | None = None)[source]

Bases: object

OAuth2-based credentials for REST API access - Phase 0 Implementation.

This credential type supports the new FFIEC REST API that uses OAuth2 Bearer tokens for authentication instead of username/password pairs.

Key Features: - OAuth2 Bearer token authentication (90-day lifecycle) - Token expiration tracking and validation - Immutable after initialization for security - Compatible with automatic protocol selection

Parameters:

username – FFIEC username (for UserID header)
bearer_token – OAuth2 bearer token (90-day lifecycle)
token_expires – Optional token expiration datetime

Example:

# Create OAuth2 credentials for REST API
creds = OAuth2Credentials(
    username="your_ffiec_username",
    bearer_token="your_90_day_bearer_token",
    token_expires=datetime(2024, 3, 15)  # Optional
)

# Use with existing methods (automatic REST API selection)
periods = collect_reporting_periods(session, creds)

property bearer_token: str

Get the OAuth2 bearer token.

Returns:: Bearer token for authentication

get_auth_headers() → Dict[str, str][source]

Get authentication headers for REST API requests.

Returns:: Dictionary containing required authentication headers

property is_expired: bool

Check if token is expired or expires within 24 hours.

Returns:: True if token is expired or expires soon, False otherwise

test_credentials(session: Session = None) → bool[source]

Test OAuth2 credentials against REST API (placeholder).

Note: This will be implemented when REST adapter is available. For now, validates token format and expiration.

Parameters:: session – Optional requests session (for future compatibility)
Returns:: True if credentials appear valid, False otherwise

property token_expires: datetime | None

Get token expiration datetime.

Returns:: Token expiration datetime or None if not set

property username: str

Get the FFIEC username.

Returns:: Username for UserID header

class ProtocolAdapter[source]

Bases: ABC

Abstract base class for API protocol adapters.

This defines the common interface that both SOAP and REST adapters must implement, ensuring consistent behavior regardless of protocol.

abstract property protocol_name: str: Return the protocol name (e.g., ‘SOAP’, ‘REST’).

abstractmethod retrieve_facsimile(rssd_id: str | int, reporting_period: str, series: str) → bytes[source]

Retrieve facsimile data for an institution.

Parameters:

rssd_id – Institution RSSD ID
reporting_period – Reporting period (MM/dd/yyyy)
series – Report series

Returns:

Raw facsimile data (XBRL bytes)

abstractmethod retrieve_filers_since_date(reporting_period: str, since_date: str) → List[str][source]

Retrieve filers who submitted since a specific date.

Parameters:

reporting_period – Reporting period (MM/dd/yyyy)
since_date – Date since which to check submissions (MM/dd/yyyy)

Returns:

List of RSSD IDs as strings

abstractmethod retrieve_filers_submission_datetime(reporting_period: str, since_date: str | None = None) → List[Dict[str, Any]][source]

Retrieve filer submission date/time information.

Parameters:

reporting_period – Reporting period (MM/dd/yyyy)
since_date – Optional date to filter submissions since (MM/dd/yyyy)

Returns:

List of submission info dictionaries

abstractmethod retrieve_panel_of_reporters(reporting_period: str) → List[Dict[str, Any]][source]

Retrieve panel of reporters for a reporting period.

Parameters:: reporting_period – Reporting period (MM/dd/yyyy)
Returns:: List of institution dictionaries

abstractmethod retrieve_reporting_periods(series: str) → List[str][source]

Retrieve available reporting periods for a series.

Parameters:: series – Report series (e.g., “call”, “ubpr”)
Returns:: List of reporting period strings (validated against schema)

abstractmethod retrieve_ubpr_reporting_periods() → List[str][source]

Retrieve UBPR reporting periods.

Returns:: List of available UBPR reporting periods

abstractmethod retrieve_ubpr_xbrl_facsimile(rssd_id: str | int, reporting_period: str) → bytes[source]

Retrieve UBPR XBRL facsimile data.

Parameters:

rssd_id – Institution RSSD ID
reporting_period – Reporting period

Returns:

Raw UBPR XBRL data bytes

class ProxyProtocol(*values)[source]

Bases: Enum

Enumerated values that represent the proxy protocol options

Parameters:: Enum (ProxyProtocol) – Enumerated value for HTTP or HTTPS

HTTP = 0

HTTPS = 1

class RESTAdapter(credentials: OAuth2Credentials, rate_limiter: RateLimiter | None = None, session: Client | None = None)[source]

Bases: ProtocolAdapter

REST API adapter for FFIEC webservice.

This adapter implements the new FFIEC REST API with OAuth2 authentication, enhanced rate limiting, and automatic data normalization to maintain backward compatibility with SOAP responses.

BASE_URL = 'https://ffieccdr.azure-api.us/public'

DEFAULT_RATE_LIMIT = 0.6944444444444444

is_rest() → bool[source]: Return True since this is a REST adapter.

property protocol_name: str: Return the protocol name.

retrieve_facsimile(rssd_id: str | int, reporting_period: str, series: str) → bytes[source]

Retrieve facsimile data via REST API.

Parameters:

rssd_id – Institution RSSD ID
reporting_period – Reporting period (MM/dd/yyyy)
series – Report series

Returns:

Raw XBRL data bytes

retrieve_filers_since_date(reporting_period: str, since_date: str) → List[str][source]

Retrieve filers who submitted since date via REST API.

Parameters:

reporting_period – Reporting period (MM/dd/yyyy)
since_date – Date since which to check submissions (MM/dd/yyyy)

Returns:

List of RSSD IDs as strings (normalized to SOAP format)

retrieve_filers_submission_datetime(reporting_period: str, since_date: str | None = None) → List[Dict[str, Any]][source]

Retrieve filer submission date/time info via REST API.

Parameters:

reporting_period – Reporting period (MM/dd/yyyy)
since_date – Optional date to filter submissions since (MM/dd/yyyy)

Returns:

List of submission info dictionaries (normalized to SOAP format)

retrieve_panel_of_reporters(reporting_period: str) → List[Dict[str, Any]][source]

Retrieve panel of reporters via REST API.

Parameters:: reporting_period – Reporting period (MM/dd/yyyy)
Returns:: List of institution dictionaries (normalized to SOAP format)

retrieve_reporting_periods(series: str) → List[str][source]

Retrieve available reporting periods via REST API.

Parameters:: series – Report series (e.g., “call”, “ubpr”)
Returns:: List of reporting period strings (normalized to SOAP format)

retrieve_ubpr_reporting_periods() → List[str][source]

Retrieve available UBPR reporting periods via REST API.

Returns:: List of reporting period strings

retrieve_ubpr_xbrl_facsimile(rssd_id: str | int, reporting_period: str) → bytes[source]

Retrieve UBPR XBRL facsimile data via REST API.

Parameters:

rssd_id – Institution RSSD ID
reporting_period – Reporting period (MM/dd/yyyy)

Returns:

Raw XBRL data bytes

exception RateLimitError(retry_after: int | None = None)[source]

Bases: FFIECError

Raised when API rate limit is exceeded

class RateLimiter(calls_per_second: float = 10)[source]

Bases: object

Thread-safe rate limiter for both sync and async use.

async async_wait_if_needed() → None[source]: Asynchronous rate limit wait.

wait_if_needed() → None[source]: Synchronous rate limit wait.

class SOAPAdapter(credentials: WebserviceCredentials, session: Session | None = None)[source]

Bases: ProtocolAdapter

SOAP API adapter wrapping existing functionality.

This adapter provides a consistent interface to the existing SOAP-based functionality, allowing for transparent protocol selection.

is_rest() → bool[source]: Return False since this is a SOAP adapter.

property protocol_name: str: Return the protocol name.

retrieve_facsimile(rssd_id: str | int, reporting_period: str, series: str) → bytes[source]: Retrieve facsimile data via SOAP (delegates to existing implementation).

retrieve_filers_since_date(reporting_period: str, since_date: str) → List[str][source]: Retrieve filers since date via SOAP (delegates to existing implementation).

retrieve_filers_submission_datetime(reporting_period: str, since_date: str | None = None) → List[Dict[str, Any]][source]: Retrieve filer submission info via SOAP (delegates to existing implementation).

retrieve_panel_of_reporters(reporting_period: str) → List[Dict[str, Any]][source]: Retrieve panel of reporters via SOAP (delegates to existing implementation).

retrieve_reporting_periods(series: str) → List[str][source]: Retrieve reporting periods via SOAP (delegates to existing implementation).

retrieve_ubpr_reporting_periods() → List[str][source]: Retrieve UBPR reporting periods via SOAP (delegates to existing implementation).

retrieve_ubpr_xbrl_facsimile(rssd_id: str | int, reporting_period: str) → bytes[source]: Retrieve UBPR XBRL facsimile via SOAP (delegates to existing implementation).

exception SessionError(message: str, session_state: str | None = None)[source]

Bases: FFIECError

Raised when there are session management issues

exception ValidationError(field: str, value: Any, expected: str)[source]

Bases: FFIECError

Raised when input validation fails

class WebserviceCredentials(username: str | None = None, password: str | None = None)[source]

Bases: object

The WebserviceCredentials class. This class is used to store the credentials for the FFIEC webservice.

Parameters:

username (str, optional) – FFIEC Webservice username. Optional: If not provided, the credentials will be set from the environment variable FFIEC_USERNAME
password (str, optional) – FFIEC Webservice password. Optional: If not provided, the credentials will be set from the environment variable FFIEC_PASSWORD

Returns:

An instance of the WebserviceCredentials class.

Return type:

WebserviceCredentials

property password: str

Returns the password from the WebserviceCredentials instance.

Returns:: the password stored in the WebserviceCredentials instance
Return type:: str

test_credentials(session: Session) → bool[source]

Test the credentials with the FFIEC Webservice to determine if they are valid and accepted.

Note: The session argument can be generated directly from requests, or

using the helper class FFIECConnection

Parameters:

session (requests.Session) – the connection to test the credentials

Returns:

True if the credentials are valid, False otherwise

Return type:

bool

Raises:

ValueError – if the credentials are not set
Exception – Other unspecified errors

property username: str

Returns the username from the WebserviceCredentials instance.

Returns:: the username stored in the WebserviceCredentials instance
Return type:: str

exception XMLParsingError(message: str, xml_snippet: str | None = None)[source]

Bases: FFIECError

Raised when XML/XBRL parsing fails

clear_soap_cache() → None[source]: Clear the global SOAP client cache.

collect_data(session: FFIECConnection | Session | None, creds: WebserviceCredentials | OAuth2Credentials, reporting_period: str | datetime, rssd_id: str, series: str, output_type: str = 'list', date_output_format: str = 'string_original', force_null_types: str | None = None) → Any[source]

Return time series data from the FFIEC webservice for a given reporting period and RSSD ID

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Translates the input reporting period to a FFIEC-formatted date Transforms the output to a pandas dataframe if output_type is ‘pandas’, otherwise returns a list

Valid arguments for the ``reporting_period` argument:

mm/dd/yyyy
yyyy-mm-dd
yyyymmdd
a python datetime object
For the above types, the date msut be the last day in the quarter (e.g. March 31, June 30, September 30, or December 31)
#Qyyyy, where # is the quarter number and yyyy is the year.

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
reporting_period (str or datetime) – Reporting period.
rssd_id (str) – The RSSD ID of the entity for which you want to retrieve data.
series (str) – call or ubpr
output_type (str) – list, pandas, or polars
date_output_format (str) – string_original, string_yyyymmdd, or python_format
force_null_types (str, optional) – Override null value handling. Options: - None (default): Automatic based on API (SOAP uses numpy, REST uses pandas) - “numpy”: Force np.nan for null values (original behavior) - “pandas”: Force pd.NA for null values (better integer handling)

Returns:

Returns data in the specified format

Return type:

list, pandas DataFrame, or polars DataFrame

Retrieves data from FFIEC webservice.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Retrieves the Financial Institutions in a Panel of Reporters for a given reporting period. Note that this function only reports on Call Report filings, not UBPR filings.

Valid arguments for the ``reporting_period` argument:

mm/dd/yyyy
yyyy-mm-dd
yyyymmdd
a python datetime object
For the above types, the date must be the last day in the quarter (e.g. March 31, June 30, September 30, or December 31)
#Qyyyy, where # is the quarter number and yyyy is the year.

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
reporting_period (str or datetime) – The reporting period to use for the request.

Returns:

List of dicts or pandas DataFrame containing the following fields: - “rssd”/”id_rssd”: Institution RSSD ID (both field names provided for compatibility) - “fdic_cert_number”: FDIC certificate number - “occ_chart_number”: OCC charter number - “ots_dock_number”: OTS docket number - “primary_aba_rout_number”: Primary ABA routing number - “name”: Institution name - “state”: State - “city”: City - “address”: Street address - “filing_type”: Filing type - “has_filed_for_reporting_period”: Whether institution has filed for the period

NOTE: Property names were inconsistent in earlier code, so both ‘rssd’ and ‘id_rssd’ are provided with identical data to reduce need to refactor existing user code.

Return type:

list or pd.DataFrame

Retrieves data from FFIEC webservice.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Retrieves the ID RSSDs of the reporters who have filed after a given date for a given reporting period. Note that this function only reports on Call Report filings, not UBPR filings.

Valid arguments for the ``since_date` argument:

mm/dd/yyyy

yyyy-mm-dd

yyyymmdd

a python datetime object

Valid arguments for the ``reporting_period` argument:

all of the above, as long as the date is the last day in the quarter (e.g. March 31, June 30, September 30, or December 31)

#Qyyyy, where # is the quarter number and yyyy is the year.

Args:
session (FFIECConnection or requests.Session): The requests session object to use for the request. creds (WebserviceCredentials): The credentials to use for the request. since_date (str or datetime): The date to use for the request. May be in the format of ‘YYYY-MM-DD’, ‘YYYYMMDD’, ‘MM/DD/YYYY’, or a python datetime object. output_type (str, optional): “list” or “pandas”. Defaults to “list”.

Returns:
any: Returns either a list of dicts or a pandas Series comprising the ID RSSDs of the reporters who have filed after a given date for a given reporting period.

collect_filers_submission_date_time(session: FFIECConnection | Session | None, creds: WebserviceCredentials | OAuth2Credentials, since_date: str | datetime, reporting_period: str | datetime, output_type: str = 'list', date_output_format: str = 'string_original') → List[Any] | DataFrame[source]

Retrieves data from FFIEC webservice.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Retrieves the ID RSSDs and DateTime of the reporters who have filed after a given date for a given reporting period. Note that this function only reports on Call Report filings, not UBPR filings.

Note on date_output_format:

string_original is the default output format, and is the format that is used by the FFIEC webservice: mm/dd/yyyy

string_yyyymmdd is the date in yyyymmdd format

python_format is the date in python datetime format

Args:
session (ffiec_connection.FFIECConnection or requests.Session): The requests session object to use for the request. creds (WebserviceCredentials or requests.Session): The credentials to use for the request. since_date (str or datetime): The date to use for the request. May be in the format of ‘YYYY-MM-DD’, ‘YYYYMMDD’, ‘MM/DD/YYYY’, or a python datetime object. reporting_period (str or datetime): The reporting period to use for the request (e.g. “2020-03-21”). Note that the date must be in the format of “YYYY-MM-DD”, “YYYYMMDD”, “MM/DD/YYYY”, #QYYYY or a python datetime object, with the month and date set to March 31, June 30, September 30, or December 31. output_type (str, optional): “list” or “pandas”. Defaults to “list”. date_output_format (str, optional): string_original or python_datetime. Defaults to “string_original”.

Returns:
any: List of dicts or pandas DataFrame containing the following fields: - “rssd”/”id_rssd”: Institution RSSD ID (both field names provided for compatibility) - “datetime”: Submission date and time in Washington DC timezone

NOTE: Property names were inconsistent in earlier code, so both ‘rssd’ and ‘id_rssd’ are provided with identical data to reduce need to refactor existing user code.

collect_reporting_periods(session: FFIECConnection | Session | None, creds: WebserviceCredentials | OAuth2Credentials, series: str = 'call', output_type: str = 'list', date_output_format: str = 'string_original') → List[str] | List[datetime] | Series[source]

Returns list of reporting periods available for access via the FFIEC webservice

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Note on date_output_format:

string_original is the default output format, and is the format that is used by the FFIEC webservice: mm/dd/yyyy
string_yyyymmdd is the date in yyyymmdd format
python_format is the date in python datetime format

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
series (str, optional) – call or ubpr
output_type (str) – list or pandas
date_output_format – string_original, string_yyyymmdd, or python_format

Returns:

Returns a list of reporting periods from the FFIEC Webservice in ascending chronological order (oldest first)

Return type:

list or Pandas series

Retrieves UBPR XBRL facsimile data for a specific institution.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
reporting_period – Reporting period date
rssd_id – Institution RSSD ID
output_type – Output format (“list”, “pandas”, “polars”, or “bytes”)
force_null_types (str, optional) – Override null value handling. Options: - None (default): Automatic based on API (REST uses pandas) - “numpy”: Force np.nan for null values - “pandas”: Force pd.NA for null values

Returns:

UBPR XBRL data

Return type:

bytes, list, or pd.DataFrame

collect_ubpr_reporting_periods(session: FFIECConnection | Session | None, creds: WebserviceCredentials | OAuth2Credentials, output_type: str = 'list', date_output_format: str = 'string_original') → List[Any] | DataFrame[source]

Retrieves UBPR reporting periods from FFIEC API.

ENHANCED: Now supports both SOAP and REST APIs automatically based on credential type. For better performance, use OAuth2Credentials for REST API access.

Parameters:

session – The session object (can be None for REST API)
creds – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
output_type – Output format (“list”, “pandas”, or “polars”)
date_output_format – Date format for output

Returns:

List of UBPR reporting periods in ascending chronological order (oldest first)

Return type:

list or pd.DataFrame

create_protocol_adapter(credentials: WebserviceCredentials | OAuth2Credentials, session: Session | None | Client = None) → ProtocolAdapter[source]

Factory function to create appropriate protocol adapter based on credential type.

This is the main entry point for automatic protocol selection. Based on the credential type provided, it returns either a SOAP or REST adapter.

Parameters:

credentials – Either WebserviceCredentials (SOAP) or OAuth2Credentials (REST)
session – Optional requests session

Returns:

Appropriate protocol adapter

Raises:

CredentialError – If credential type is not supported

disable_legacy_mode() → None[source]: Disable legacy compatibility mode and use new features.

enable_legacy_mode() → None[source]

Enable full legacy compatibility mode.

This enables: - Legacy ValueError exceptions instead of specific exception types - Original error messages without additional context

get_cache_stats() → Dict[str, Any][source]: Get statistics about the SOAP client cache.

set_legacy_errors(enabled: bool) → None[source]: Set whether to use legacy error behavior.

use_legacy_errors() → bool[source]: Check if legacy error mode is enabled.