FEAT: Implementing and Integrating AQE API's - execute, executemany, fetch, fetchall and fetchmany - #792
FEAT: Implementing and Integrating AQE API's - execute, executemany, fetch, fetchall and fetchmany#792Subrata (subrata-ms) wants to merge 8 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved moderate and critical findings remain in result-set state, parameter binding, fetch handling, and credential-redaction coverage.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Refactors the async query API by separating execution and fetching helpers, improving error translation and row handling, and expanding test coverage.
Changes:
- Extracts async execution and fetch operations into dedicated modules.
- Improves exception translation, logging, row wrapping, and fetch tracking.
- Expands tests for execution, fetching, errors, cursor behavior, and logging.
File summaries
| File | Reviewed changes | Final review findings |
|---|---|---|
tests/AsyncTest/test_007_async_fetch.py |
Adds fetch, row, type, and navigation coverage. | — |
tests/AsyncTest/test_006_async_execute.py |
Adds execute and executemany coverage. | — |
tests/AsyncTest/test_005_async_cursor.py |
Updates cursor property and lifecycle tests. | — |
tests/AsyncTest/test_004_async_logging.py |
Adds operation logging coverage. | Critical (2 votes): Preserve broad credential-redaction assertions or verify known secrets are absent. |
tests/AsyncTest/test_003_async_exceptions.py |
Expands exception translation coverage. | — |
tests/AsyncTest/test_002_async_connection.py |
Updates connection error and lifecycle tests. | — |
mssql_python/async_query/exception_translator.py |
Classifies and translates async errors. | — |
mssql_python/async_query/async_fetch.py |
Implements fetching, row wrapping, and metadata handling. | Moderate (1 vote): Snapshot native_uuid; short-circuit non-positive fetch sizes. Nit (3 votes): Cache row metadata maps instead of rebuilding them per row. |
mssql_python/async_query/async_execute.py |
Implements async execution helpers. | Moderate (2 votes): Normalize Row parameter sequences like the synchronous path. |
mssql_python/async_query/async_cursor.py |
Delegates operations and tracks fetch state. | Moderate (2 votes): Snapshot lowercase/UUID settings and row metadata per result set. |
mssql_python/async_query/async_connection.py |
Renames connection internals and integrates logging. | — |
Review details
Suppressed comments (2)
mssql_python/async_query/async_fetch.py:62
- The wrapper passes a non-positive
sizethrough to py-core. The synchronous cursor short-circuitssize <= 0before its native call (mssql_python/cursor.py:2842-2843), and this PR's test requiresfetchmany(-1)to return[]; a Rust binding using an unsigned size can reject-1instead. Short-circuitrequested_size <= 0before calling py-core.
with translate_py_core_exceptions():
if size is None:
rows = await _get_py_core_async_cursor(cursor).fetchmany()
else:
rows = await _get_py_core_async_cursor(cursor).fetchmany(size)
mssql_python/async_query/async_fetch.py:30
- This consults the mutable global
native_uuidat fetch time, so changing it betweenexecute()andfetch*()changes the representation of an already-executed result. The synchronous cursor snapshots UUID conversion indices at execute (mssql_python/cursor.py:1404-1423) and tests cover this (tests/test_004_cursor.py:16785-16815); store the setting/result-set metadata when execute or nextset completes.
uuid_str_indices = (
tuple(index for index, column in enumerate(description) if column[1] is uuid.UUID)
if not get_settings().native_uuid
else None
- Files reviewed: 11/11 changed files
- Comments generated: 4
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| lowercase = get_settings().lowercase | ||
| return [ | ||
| ((column[0].lower() if lowercase else column[0]), *column[1:]) for column in description |
| if len(parameters) == 1 and isinstance(parameters[0], (tuple, list)): | ||
| parameters = tuple(parameters[0]) |
| def _wrap_row(cursor: "AsyncCursor", values: tuple[Any, ...]) -> Row: | ||
| description = cursor.description or () | ||
| column_map = {column[0]: index for index, column in enumerate(description)} | ||
| column_map_lower = ( | ||
| {name.lower(): index for name, index in column_map.items()} |
📊 Code Coverage Report
Diff CoverageDiff: main...HEAD, staged and unstaged changes
Summary
📋 Files Needing Attention📉 Files with overall lowest coverage (click to expand)mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 61.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.row.py: 77.6%
mssql_python.pybind.ddbc_bindings.cpp: 77.7%
mssql_python.pybind.connection.connection_pool.cpp: 81.8%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.py_type_cache.hpp: 91.6%🔗 Quick Links
|
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved compatibility, correctness, and performance findings remain.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (4)
mssql_python/async_query/async_cursor.py:107
- This property applies the global
lowercasesetting on every access, so changing it afterexecute()changes the metadata and row key casing for an already-established result set. The synchronous cursor snapshots this at execution (tests/test_004_cursor.py:3592-3608); retain the effective casing with the async result metadata instead.
if description is None:
return None
lowercase = get_settings().lowercase
return [
((column[0].lower() if lowercase else column[0]), *column[1:]) for column in description
mssql_python/async_query/async_execute.py:27
- The new async path only unwraps tuple/list parameters, but the added
test_execute_accepts_dbapi_rowpasses aRowand expects its columns to bind as individual parameters. The synchronous implementation explicitly normalizesRowto a tuple because the downstream binder otherwise treats the whole row as one value (mssql_python/cursor.py:1739-1747); apply the same normalization before forwarding toPyAsyncCursor.execute, or that new async contract will fail.
cursor._reset_fetch_tracking() # pyright: ignore[reportPrivateUsage]
if len(parameters) == 1 and isinstance(parameters[0], (tuple, list)):
parameters = tuple(parameters[0])
mssql_python/async_query/async_fetch.py:30
native_uuidis read while each row is wrapped, so toggling the module setting afterexecute()but beforefetch*()changes the type of rows from one result set. The synchronous cursor snapshots this setting at execution (tests/test_004_cursor.py:16336-16357); cache the UUID conversion indices when async result metadata is established.
uuid_str_indices = (
tuple(index for index, column in enumerate(description) if column[1] is uuid.UUID)
if not get_settings().native_uuid
else None
mssql_python/async_query/async_fetch.py:23
_wrap_rowrebuilds the column map and, when enabled, the lowercase map for every returned row.fetchmany()andfetchall()therefore repeat O(column_count) metadata work for every row; the synchronous cursor precomputes these maps once per result set, so cache and reuse them for large async result sets.
def _wrap_row(cursor: "AsyncCursor", values: tuple[Any, ...]) -> Row:
description = cursor.description or ()
column_map = {column[0]: index for index, column in enumerate(description)}
column_map_lower = (
{name.lower(): index for name, index in column_map.items()}
- Files reviewed: 11/11 changed files
- Comments generated: 1
- Review effort level: Lite
| seq_of_parameters: Sequence[Sequence[Any]] | Sequence[Mapping[str, Any]], | ||
| ) -> None: | ||
| await async_execute.executemany( | ||
| self, | ||
| operation, |
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
There was a problem hiding this comment.
🟡 Changes recommended
A critical test indentation error blocks collection, and fetch behavior, performance, and documentation require fixes.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (3)
mssql_python/async_query/async_fetch.py:24
_wrap_rowis called for every row byfetchone,fetchmany, andfetchall, but this rebuilds the description-derived maps for every row, including a freshcursor.descriptionlist and dictionaries. That makes large result sets pay O(rows × columns) Python metadata work and can make one result set change representation iflowercase/native_uuidis changed between fetches; the synchronous cursor caches these maps/settings once per result set (mssql_python/cursor.py:1838-1848). Cache the row metadata when the result set is established and reuse it here.
description = cursor.description or ()
column_map = {column[0]: index for index, column in enumerate(description)}
column_map_lower = (
{name.lower(): index for name, index in column_map.items()}
if get_settings().lowercase
mssql_python/async_query/async_fetch.py:60
- Non-positive sizes are still passed to the native cursor. The synchronous cursor returns immediately for
size <= 0(mssql_python/cursor.py:2839-2843), and the new async test expectsfetchmany(-1)to return[]; passing-1to a nativeusize-like parameter can instead raise before the wrapper's tracking guard runs.
requested_size = cursor.arraysize if size is None else size
logger.debug("AsyncCursor.fetchmany: starting; requested_size=%s", requested_size)
with translate_py_core_exceptions():
if size is None:
rows = await _get_py_core_async_cursor(cursor).fetchmany()
mssql_python/async_query/exception_translator.py:98
- The new fallback at this line translates allowlisted built-in
RuntimeErrorandTypeErrorinstances, but the surrounding docstrings still say non-py-core errors are returned unchanged and that the context manager translates only py-core failures. Please update that documented contract so callers are not misled about which native Python errors are wrapped.
return _translate_known_builtin_error(error)
- Files reviewed: 11/11 changed files
- Comments generated: 1
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
Five moderate issues remain unresolved.
Review details
Suppressed comments (5)
Previously missed (1) — in code that hasn't changed since the last review.
mssql_python/async_query/async_fetch.py:62
- The synchronous cursor returns
[]immediately forsize <= 0(mssql_python/cursor.py:2839-2843), but this path passes zero and negative sizes intomssql_py_core. That makes the newfetchmany(0/-1)contract depend on backend validation and can raise instead of returning the expected empty list; short-circuit non-positive sizes before the native call.
mssql_python/async_query/async_cursor.py:87
- After a fetch sets
_fetch_rowcount,rowcountbypasses the native cursor on every later read. Becauseclose()does not clear that cache, a fetched cursor still returns its old count after closing instead of following the native closed-cursor behavior used by the other properties and operations. Clear fetch tracking when close succeeds before logging completion.
await self._py_core_async_cursor.close()
mssql_python/async_query/async_cursor.py:62
- This removes the existing public
use_preparekeyword fromAsyncCursor.executemany; callers that used the previous signature now fail withTypeError, and the native helper no longer receives the flag. Retain and forward the keyword (asexecutestill does), or deliberately version/document this breaking API change.
seq_of_parameters: Sequence[Sequence[Any]] | Sequence[Mapping[str, Any]],
) -> None:
mssql_python/async_query/async_execute.py:27
- This unwraps only tuples/lists, so a
Rowpassed as the sole parameter is forwarded toPyAsyncCursor.executeas one bound value. The newtest_execute_accepts_dbapi_rowexpects the row's two values to bind to two?markers; mirror the synchronous cursor's Row-to-tuple normalization (mssql_python/cursor.py:1742-1747) before calling the native API.
if len(parameters) == 1 and isinstance(parameters[0], (tuple, list)):
parameters = tuple(parameters[0])
mssql_python/async_query/async_fetch.py:30
_wrap_rowruns once for every returned row, but each invocation rereadsdescription, rebuilds both column maps, and rescans all columns for UUIDs. Largefetchall()/fetchmany()results therefore repeat O(column_count) metadata work and allocate a map per row; cache these result-set maps onAsyncCursorwhen execution/nextset()changes and reuse them here.
description = cursor.description or ()
column_map = {column[0]: index for index, column in enumerate(description)}
column_map_lower = (
{name.lower(): index for name, index in column_map.items()}
if get_settings().lowercase
else None
)
uuid_str_indices = (
tuple(index for index, column in enumerate(description) if column[1] is uuid.UUID)
if not get_settings().native_uuid
else None
- Files reviewed: 11/11 changed files
- Comments generated: 0 new
- Review effort level: Lite
Work Item / Issue Reference
Summary
This pull request refactors the asynchronous query layer to improve clarity, error handling, and code organization. The main changes include renaming internal variables for clarity, extracting statement execution and result fetching logic into dedicated modules, enhancing exception translation, and updating tests to reflect improved error handling. These updates make the async API more robust and maintainable.
Refactoring and Code Organization:
native_connection/native_cursortopy_core_async_connection/py_core_async_cursorin bothAsyncConnectionandAsyncCursorfor improved clarity and future maintainability. (mssql_python/async_query/async_connection.py,mssql_python/async_query/async_cursor.py) [1] [2]AsyncCursorinto new helper modules:async_execute.pyandasync_fetch.py, leading to cleaner, more modular code. (mssql_python/async_query/async_execute.py,mssql_python/async_query/async_fetch.py,mssql_python/async_query/async_cursor.py) [1] [2] [3]Error Handling Improvements:
exception_translator.pyto classify and translate lower-level driver errors into appropriate public exceptions (e.g.,OperationalError,ProgrammingError), and updated the async connection tests to verify these translations. (mssql_python/async_query/exception_translator.py,tests/AsyncTest/test_002_async_connection.py) [1] [2] [3] [4]Behavioral and API Improvements:
AsyncCursor, including accuraterowcountreporting after fetch operations and resetting fetch state onnextset. (mssql_python/async_query/async_cursor.py) [1] [2]AsyncCursor.descriptionto support automatic lowercasing based on settings, and ensured that row wrapping consistently producesRowobjects with proper metadata. (mssql_python/async_query/async_cursor.py,mssql_python/async_query/async_fetch.py) [1] [2]Testing:
tests/AsyncTest/test_002_async_connection.py,tests/AsyncTest/test_003_async_exceptions.py) [1] [2] [3]