Opened 2 years ago
Last modified 4 months ago
#28434 new defect
Meta-ticket: Callable symbolic expressions
Reported by: | rburing | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.5 |
Component: | symbolics | Keywords: | CallableSymbolicExpression, function, callable |
Cc: | slelievre, mjo | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
The syntax for callable symbolic expressions works in the simplest cases, but beyond that it breaks down and causes much confusion, as can be often seen on Ask SageMath and (in my experience) when trying to teach SageMath to people. Here is a non-exhaustive list of 8 examples:
- Numbers confusion:
sage: f(x) = x^2 sage: f(2).factor() 4
- Polynomial confusion:
sage: R.<z,w> = PolynomialRing(QQ) sage: f(x) = x^2 sage: f(z+w).coefficient({z : 1}) ... TypeError: no canonical coercion from <type 'dict'> to Symbolic Ring
https://ask.sagemath.org/question/47064/how-to-turn-the-function-into-expression/
- Matrix confusion:
sage: B(x) = matrix([[x, 0], [0, 0]]) sage: B(12) [x 0] [0 0]
https://ask.sagemath.org/question/10457/arithmetic-with-matrices-of-formal-functions/
- List confusion:
sage: f(x) = [x,x] sage: f(2).parent() Vector space of dimension 2 over Symbolic Ring
https://ask.sagemath.org/question/10449/how-to-return-a-list-from-callable-symbolic-expression/
- Derivative confusion (and argument confusion):
sage: f(x) = x.derivative() sage: f(x^2) 1 sage: f(y) = y.derivative(x) sage: f(x^2) 0
- Matrix argument confusion:
sage: f(x) = x^2 sage: f(2*identity_matrix(2)) ... TypeError: no canonical coercion from Full MatrixSpace of 2 by 2 dense matrices over Integer Ring to Callable function ring with argument x
https://ask.sagemath.org/question/38524/defining-functions-acting-on-matrix-elements/
- Addition confusion - #32008:
sage: f(x) = x^2 sage: g(x) = x^2 sage: var('t') sage: h(t) = t^2 sage: f+g x |--> 2*x^2 sage: f+h (t, x) |--> t^2 + x^2
https://ask.sagemath.org/question/10782/symbolic-functions-without-named-variables/
- Non-symbolic function confusion:
sage: tau(n) = len(divisors(n)) ... TypeError: 'sage.symbolic.expression.Expression' object is not iterable
https://ask.sagemath.org/question/47672/is-this-a-bug-or-intended-behavior/
- Derivative confusion.
sage: f(t) = (t, t^2, t^3) sage: f t |--> (t, t^2, t^3) sage: f(t) (t, t^2, t^3) sage: f.parent() Vector space of dimension 3 over Callable function ring with argument t sage: f(t).parent() Vector space of dimension 3 over Symbolic Ring sage: f1 = f(t).derivative(t) sage: f1 (1, 2*t, 3*t^2) sage: f1.parent() Vector space of dimension 3 over Symbolic Ring sage: g(t) = f(t).derivative(t) Traceback (most recent call last) ... TypeError: unable to convert (1, 2*t, 3*t^2) to a symbolic expression sage: h = f.derivative() sage: h [ t |--> 1] [ t |--> 2*t] [t |--> 3*t^2] sage: h.parent() Full MatrixSpace of 3 by 1 dense matrices over Callable function ring with argument t
sage: f(t) = (t, ....: t^2, ....: t^3) File "<ipython-input-46-d381d3086d21>", line 2 t**Integer(2), ^ SyntaxError: invalid syntax
Additional issues and tickets related to callable symbolic expressions:
- #7512 fast_callable should respect the variable order in callable symbolic expressions (treating them like lambda functions rather than like expressions)
- #11507 make f(x,y,z)=vector make a vector-valued function
- #12075 Create callable matrices in function notation
- #12302 Partial evaluation bug for callable vector functions
- #15219 numerical integral needs an operand for callable symbolic functions
- #32008
CallableSymbolicExpressionRing
over subrings ofSR
- #32146
CallableSymbolicExpressionRing
: Fix broken element methods - #30374 Improve repr for callable symbolic expressions using unicode
- #31282 Allow to take the sign of a callable symbolic expression
- #30953 Implicit line continuation in callable symbolic expression
Change History (19)
comment:1 Changed 2 years ago by
- Description modified (diff)
comment:2 follow-up: ↓ 3 Changed 2 years ago by
comment:3 in reply to: ↑ 2 ; follow-up: ↓ 4 Changed 2 years ago by
I have to disagree that these issues are unrelated. The main source of confusion (in my opinion) is that this syntax is reserved for something very particular (callable symbolic expressions), while it looks like something much more general, like a lambda-style definition of a Python function.
- The most obvious bug IMHO. Factoring in SR, especially in obvious cases like this, should work. This has nothing to do with callable expressions though.
What I meant to show was: the (naive) expectation is that when you put in an integer, you get back an integer. This is false, because what you get is a symbolic expression. So a better example (with a method that isn't available on symbolic expressions) would be f(2).binary()
.
- Not really a bug, you shouldn't mix symbolic expressions and polynomials.
They can often be mixed. A "fix" here is to write R(f(z+w)).coefficient({z : 1})
. My point is: how does the user know that f(x) = x^2
defines a callable symbolic expression which should get this special treatment (compared to a Python function)? Or, how should a user know not to use these in this context?
- Not a bug, you cannot make a list symbolic.
How does the user know that this is an issue? It's not an issue with Python functions...
- Not a bug, the RHS
x.derivative()
equals 1 sof(x) = x.derivative()
is just a complicated way to writef(x) = 1
.I agree that this may be confusing, but there really is an important difference between
f(x) = x.derivative()and
def f(x): return x.derivative()
We know this and we are used to it, but in my opinion it really makes no sense. I can kind of understand the desire to create a type of function object (which is what happens in the first case), e.g. for prettyprinting, but the fact that these two f
's don't give the same output when evaluated is honestly ridiculous. How should a user know?
- I don't see the problem here, what would you expect?
Some would expect two scalar-valued functions of a scalar variable to always add up to a scalar-valued function of a scalar variable. But I will concede this is the least confusing one.
- Not a bug, similar to 5: the RHS
divisors(n)
is meaningless for a symbolicn
.
How does the user know that they are defining a symbolic callable expression, and that they shouldn't?
All the questions were rhetorical. In my experience (and in the posted threads), the user doesn't know, because it isn't clear. Sage would be less confusing without this syntax (in its current form).
comment:4 in reply to: ↑ 3 Changed 2 years ago by
All the questions were rhetorical. In my experience (and in the posted threads), the user doesn't know, because it isn't clear. Sage would be less confusing without this syntax (in its current form).
So, is this ticket suggesting to remove the syntax? If so, I'm not convinced a ticket is the right place to have such a discussion.
Do you realize how much user code it would break? Maybe you should lay out your plan more clearly on sage-devel.
comment:5 follow-ups: ↓ 6 ↓ 8 Changed 2 years ago by
Thanks for putting the list together. It's nice to have a record which issues (often?) arise. A lot of confusion happens because of the mathematics underneath. Confusion is a common state for students who are learning mathematics and I think it is to be expected that they will also encounter it while working with a computer algebra system of significant scope and complexity. It's nice to reduce confusion where possible, but I think it's unrealistic to expect we can do away with it.
Some of your items are indicated above as bugs and can probably be solved. Other items might be better to put in an FAQ document (or the documentation) for people to point to if questions arise. I don't know if these examples are best kept track of on a ticket. The ticket itself isn't really resolvable, but there are other administrative tracker tickets around.
Answer to "how does a user know"?
sage: f(x)=x^2 sage: type(f) <type 'sage.symbolic.expression.Expression'> sage: parent(f) Callable function ring with argument x
Once a user run into issues like this, it's clear that they have to progress beyond the level of "beginner" and start a more "advanced" approach. When they have questions concerning this, they have a motivation to read an "advanced" tutorial that goes more into the nuts and bolts of things. After that tutorial they should be at a level where they can make sense of the reference guide. Perhaps the "advanced" tutorial already exists. Perhaps it needs to be written?
Note that callable symbolic expressions serve a very definite purpose and are therefore not going away. The preparse trick of defining them f(x)=...
is magic but incredibly useful for giving compact illustrations in educational settings, and that syntax doesn't seem to be a source for confusion anyway (other than that people don't get much of a cue that this is not a python built-in feature -- but letting people look at preparse("...")
quickly illustrates what's going on.
Callable symbolic expressions are necessary to use positional call syntax for symbolically defined functions/expressions, which is very common in math text books.
A general comment: Building a computer algebra system on top of python is necessarily a compromise. This is true for other python-based solutions too, for instance pandas or matplotlib. In most of these cases, a Domain Specific Language would be much more elegant and unified. Although, beware what you ask for: in for instance Maple, they DO have a very clear symbolic expression model that is fundamental to everything. And they found it doesn't address all their needs and had to bolt on other solutions. It makes for a decidedly unpleasant system to program for. They ended up with their compromises in other places.
1) symbolic factor COULD try and use integer factorization on objects that look like an integer or a rational number. That would change the complexity of the routine significantly (but perhaps for SR we don't care), and of course it might make people wonder why 2*I doesn't get factored.
5) doesn't seem to have to do with callable expressions at all. The fact that "derivative" is allowed to try to do something without an argument is necessary because of univariate calculus
6) Bug maybe, but possibly a consequence of design that will be hard to fix: "symbolic matrices" aren't members of the symbolic ring. They are matrices *over* SR. So filling them into a function that is from and to SR is problematic.
comment:6 in reply to: ↑ 5 ; follow-up: ↓ 7 Changed 2 years ago by
Note that callable symbolic expressions serve a very definite purpose and are therefore not going away. The preparse trick of defining them
f(x)=...
is magic but incredibly useful for giving compact illustrations in educational settings, and that syntax doesn't seem to be a source for confusion anyway (other than that people don't get much of a cue that this is not a python built-in feature -- but letting people look atpreparse("...")
quickly illustrates what's going on.
Thanks for a very well-organized resume of the issues at stake, Nils.
comment:7 in reply to: ↑ 6 Changed 2 years ago by
comment:8 in reply to: ↑ 5 Changed 2 years ago by
Thanks, Nils.
I had a look at the implementation. A callable symbolic expression like f(x) = x^2
is mostly just an Expression
whose parent is a CallableSymbolicExpressionRing
; it keeps track of the list of (symbolic variable) arguments and implements _call_element
(for substitution) and _repr_element
(for prettyprinting). The notation f(x,y) = (x,y)
defines a vector over a CallableSymbolicExpressionRing
, and the special element class Vector_callable_symbolic_dense
is used to fix the notation with a _repr_
method.
This is quite simple and powerful, and I now agree that the notation should be kept. It seems that the best resolution is to fix the bugs, improve the documentation, improve error handling, and extend the functionality to include a matrix-valued variant. See a detailed proposal below.
I found that the Guided Tour already has a page Some Common Issues with Functions which is a great start for clarifying some of the issues. This should be the page to point people to when they run into any of this.
Let me go over the list again:
1) If
f(x) = x^2
thenf(2)
is not anInteger
The Common Issues page point 2 should explain more precisely what a callable symbolic expression is. In particular, at definition-time, the arguments are symbolic variables (and treated as such on the right-hand side); calling means substitution, and the result (in the scalar-valued case) is a symbolic expression. Also, the "vector-valued" (and possibly "matrix-valued", see 3 below) variant deserves a mention.
2) Polynomial confusion
Covered by the previous point. I would like to have the "solution" to this (explicit conversion) as an example somewhere, but since the Common Issues page is part of the Guided Tour it would be too early there.
3) Matrix confusion
Currently a bug. I believe matrices are not much different from vectors, and a matrix-valued variant could be implemented. It should return a matrix over a CallableSymbolicExpressionRing
, and a special element class with _repr_
and _latex_
(like Vector_callable_symbolic_dense
) should fix the notation.
4) List confusion
In my opinion the syntax with square brackets for vector-valued functions is an abuse of notation (which has caused confusion with lists); it should be f(x) = (x,x)
or f(x) = vector([x,x])
(the latter currently doesn't work, by the way). This is handled in symbolic_expression
.
5) Derivative confusion (and argument confusion)
Handled by point 1. In particular, the point is that the arguments in the right-hand side of the definition of a callable symbolic expression are symbolic variables, not arbitrary objects, and in particular they cannot be functions.
6) Matrix argument confusion
From the implementation we know that _call_element
is a substitution of symbolic variables. So all arguments must be (coerced to) symbolic expressions. Since this is not possible for a matrix, we can give a much better error here.
7) Addition confusion
I can live with this. Maybe it should be mentioned in the documentation.
8) Non-symbolic function confusion
Can we improve the preparser here? Currently, it translates tau(n) = len(divisors(n))
into
__tmp__=var("n"); tau = symbolic_expression(len(divisors(n))).function(n)
But how about this instead?
__tmp__=var("n"); try: tau = symbolic_expression(len(divisors(n))).function(n) except Exception, e: raise TypeError, "Failed to define callable symbolic expression. Is the right-hand side a valid (tuple/vector/matrix of) symbolic expression(s) in symbolic variable(s) " + str(__tmp__) + "? Original exception: " + str(e)
The best part about this is that it informs the user that it is attempting to define a callable symbolic expression. If this is not what they meant, the user will probably do a web search for "Sage function" and will land on the (highly ranked) Common Issues page, which starts with the definition of ordinary Python functions.
comment:9 Changed 2 years ago by
- Milestone changed from sage-8.9 to sage-9.1
Ticket retargeted after milestone closed
comment:10 Changed 20 months ago by
- Milestone changed from sage-9.1 to sage-9.2
Batch modifying tickets that will likely not be ready for 9.1, based on a review of the ticket title, branch/review status, and last modification date.
comment:11 Changed 15 months ago by
- Milestone changed from sage-9.2 to sage-9.3
comment:12 Changed 10 months ago by
- Milestone changed from sage-9.3 to sage-9.4
Setting new milestone based on a cursory review of ticket status, priority, and last modification date.
comment:13 Changed 8 months ago by
- Cc slelievre added
Suggestion: make this a meta-ticket and track each item in its own ticket.
- Derivative confusion.
sage: f(t) = (t, t^2, t^3) sage: f t |--> (t, t^2, t^3) sage: f(t) (t, t^2, t^3) sage: f.parent() Vector space of dimension 3 over Callable function ring with argument t sage: f(t).parent() Vector space of dimension 3 over Symbolic Ring sage: f1 = f(t).derivative(t) sage: f1 (1, 2*t, 3*t^2) sage: f1.parent() Vector space of dimension 3 over Symbolic Ring sage: g(t) = f(t).derivative(t) Traceback (most recent call last) ... TypeError: unable to convert (1, 2*t, 3*t^2) to a symbolic expression sage: h = f.derivative() sage: h [ t |--> 1] [ t |--> 2*t] [t |--> 3*t^2] sage: h.parent() Full MatrixSpace of 3 by 1 dense matrices over Callable function ring with argument t
sage: f(t) = (t, ....: t^2, ....: t^3) File "<ipython-input-46-d381d3086d21>", line 2 t**Integer(2), ^ SyntaxError: invalid syntax
comment:14 Changed 5 months ago by
For item 7, see #32008.
comment:15 Changed 4 months ago by
- Milestone changed from sage-9.4 to sage-9.5
comment:16 Changed 4 months ago by
- Cc mjo added
- Description modified (diff)
- Summary changed from Syntax for callable symbolic expressions causes too much confusion to Meta-ticket: Callable symbolic expressions
comment:17 Changed 4 months ago by
- Description modified (diff)
comment:18 Changed 4 months ago by
- Description modified (diff)
comment:19 Changed 4 months ago by
- Description modified (diff)
Let's look at these one by one:
x.derivative()
equals 1 sof(x) = x.derivative()
is just a complicated way to writef(x) = 1
.divisors(n)
is meaningless for a symbolicn
.Some of these (in particular 4, 5, 8) are really user mistakes where an ordinary Python function is meant instead of a symbolic function. I agree that this may be confusing, but there really is an important difference between
and
I recommend you to create separate tickets for the bugs that you want to see fixed. I cannot at all promise that they will be fixed. But it will help discussion if we don't have a single issue which is about multiple unrelated things.