Skip to content

better handle quoted columns and tables#26

Open
djerius wants to merge 2 commits intodamil:masterfrom
djerius:quote
Open

better handle quoted columns and tables#26
djerius wants to merge 2 commits intodamil:masterfrom
djerius:quote

Conversation

@djerius
Copy link
Copy Markdown
Contributor

@djerius djerius commented Aug 4, 2025

Tables and columns were not properly quoted in all instances when the
quote_char constructor option is set. Some were underquoted, some over quooted,
and some table expressions were quoted. This lead iin the latter cases to illegal SQL.

The intrinsic problem lies with treating them as strings, rather than
as objects, so there is no "quote" or "quotable" state assigned to a
given SQL fragment. The code has to rely upon heuristics to know when
to quote (e.g. a bare identifier) and when not to quote (interpolation
of an SQL expression). This requires carrying external state (e.g.,
the new is_literal flag from _parse_table and in the return from
table_alias), and informing the quoting apparati in table_alias() and
column_alias() of whether quoting is permissible via the "quote_name"
and "quote_aliased_name" options.

djerius added 2 commits August 4, 2025 14:11
change from double to single quotes in 01-sql_abstract_more_quoted.t
to make it easier to compare with 08-sql_abstract_more_quoted.t

add test for literal sql in joins with  using()

add test for CTE
Tables and columns were not properly quoted in all instances when the
quote_char constructor option is set.  Some were underquoted, some over quooted,
and some table expressions were quoted.  This lead iin the latter cases to illegal SQL.

The intrinsic problem lies with treating them as strings, rather than
as objects, so there is no "quote" or "quotable" state assigned to a
given SQL fragment.  The code has to rely upon heuristics to know when
to quote (e.g. a bare identifier) and when not to quote (interpolation
of an SQL expression).  This requires carrying external state (e.g.,
the new is_literal flag from _parse_table and in the return from
table_alias), and informing the quoting apparati in table_alias() and
column_alias() of whether quoting is permissible via the "quote_name"
and "quote_aliased_name" options.
@damil
Copy link
Copy Markdown
Owner

damil commented Mar 31, 2026

Hi djerius, thanks for your contributions, I apologize for remaining silent for so long.

You pointed out various problems with quote_char. Indeed that support is not satisfactory, and I must confess that I never invested much in that matter because I did not use that option in my professional environment.
I am not so keen in adding complexity in SQL::Abstract::More for getting around problems that originated mostly in SQL::Abstract::Classic and SQL::Abstract 2.0, and at that time I wanted to be compatible with both. Now the situation has changed because both SQL::Abstract::Classic and SQL::Abstract 2.0 are no longer maintained; so I'm considering a new strategy where I would drop the dependencies on those parent classes, and fork some of SQL::Abstract::Classic code into SQL::Abstract::More. Then I would have more freedom to properly handle the 'quote_char' option.
Any thoughts on that strategy?

@djerius
Copy link
Copy Markdown
Contributor Author

djerius commented Mar 31, 2026

Hi; no worries, life is more than software. I'm afraid my reply below is somewhat disjointed.

I think there's a fundamental problem with the SQL::Abstract approach, which is that it uses heuristics to guess at how things should be rendered as SQL, providing escapes for when its model can't handle the complexities. It doesn't provide a way of attaching types to tokens, so quoting issues become corner cases which aren't always easily handled.

Your approach in SQL::Abstract::More is I think more sane, but relying upon SQL::Abstract makes it vulnerable to the same issues.

Bringing SQL::Abstract into SQL::Abstract::More would allow you more freedom to innovate. However, if there is some plan amongst the Perl database community to maintain it, that would dilute resources. I don't think the community has the resources to sustain two SQL generators. I am concerned we'd once more be in a position (as was the case with DBIx::Class, and now SQL::Abstract) where there's a sole developer with no redundant community support.

There's definitely a need for a common SQL generator. While there are a number on CPAN, by far the most used are SQL::Abstract::More and SQL::Abstract. My anecdotal experience is that SQL::Abstract::More is the choice of newer software due its increased functionality.

I think this is something which deserves a wider discussion; I don't think the burden should fall solely on your shoulders, and I'm not sure what the venue should be. (Despite my interest, my use of databases is secondary (tertiary) to my primary work, and I deal with very tiny databases on the modern scheme of things, so I have very little real-world knowledge, other than having a knack for finding corner cases and being drawn to shiny new things).

BTW, something which caught my eye recently is SQL::Wizard,. I haven't done more than scan the documentation, so can't say anything of note other than it does seem verbose.

@damil
Copy link
Copy Markdown
Owner

damil commented Mar 31, 2026

Thanks for your answer.
I didn't know SQL::Wizard, this looks like a promising step in the direction you are advocating (representing the SQL as an abstract tree internally). This was also the goal of Matt Trout in his v2.0 of SQLA, but he also had the ambition to be 100% compatible with old SQLA, so it took him years because it was very complex to try to reconcile both. When v2.0 finally came out I'm afraid that nobody ever did anything with the new internal structure.

Perl has more than one way to do it for every subject, including databases. DBI is quite universal, but on top of that there are dozens of ORMs. I tried to build a community around my DBIx::DataModel but it didn't work very well, most people preferred DBIx::Class.
Regarding SQL::Abstract::More, I'm just thinking of some modest improvements, while keeping compatibility with DBIx::DataModel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants