I mostly live in the Rust world, not the Python world, but I worked on two Python packages wrapping over my own Rust libraries and it's been pretty great.
What is your point? (I honestly do not understand.)
Blackbear's comment was that writing libraries in C allows those libraries to be deployed broadly in many compute environments.
Jakob's reply (as I understood it) was that outside of the big Deep Learning libraries, this has not really happened. There is no C implementation of Pandas that allows for redeployment in other non-python compute contexts.
My point was that, with Arrow, this type of cross platform compatibility is coming to python dataframe libraries. You can prototype Dask code that runs on your laptop, then deploy it to a production Spark cluster, knowing the same Arrow engine is underpinning both. Or at least that's the vision. Obviously Arrow is still relatively young. But the point is, it's far from certain that the long-term global optimum for the ecosystem isn't sticking with "all libraries are written in C".
In response to "rise of," I too was excited about Arrow until I played with it and realized it didn't even provide a shape attribute. Anyways, people shouldn't be dependent on a low level lang like C to write fast code.
Fair. I agree Arrow is still more of a vision than anything else.
> it didn't even provide a shape attribute
I suspect this has to do with the project's focus. I think they aspire to be a back-end to DataFrame libraries, which are generally 2d. I think they (correctly) are ceding the "n-dimensional tensor computation" space to the current incumbents.
Arrow is getting support for N-d arrays, so if anything they're expanding in that area (which is exciting). I don't think they're interested in creating a universal libarrow though, the point of the data format and C data interface is to have languages define their own implementations.
For sure, I didn't mean to imply they weren't looking at compute too! https://github.com/apache/arrow-datafusion is another example of the shared compute vision. What I was trying to point out is that (at least for Arrow core) they seem to eschew FFI and generating shared libraries in favour of from scratch implementations in other compiled languages and direct bindings in interpreted ones.
> for library writers, how many people might use it can be a deciding factor on investing the time
Yes. Especially given that a high percentage of library development these days is being done by industry, not just academia. (At least in machine learning.) They're not writing packages for fun or for academic citations, they want users.
Julia does appear to have a higher than usual proportion of package authors to total users, FWIW. I think it's both a blessing and a curse, because feedback around gaps in the ecosystem is sometimes brushed away with "write your own package". That is, because a large part of the community writes packages to scratch their own itch (I've yet to see one written for citations), they expect others to do the same (which may not be fair).
If the next machine learning killer-app model requires autodiff'ed dynamic control flow, do you think Google/Facebook will build that capability into XLA/TorchScript? Seems like if NLP SOTA requires dynamic control flow, Google will build it? Maybe they let you declare some subgraph as "dynamic" to avoid static optimizations? But maybe the static graph assumption is so deeply embedded into the XLA architecture, they'd be better off just adopting Julia? (I honestly don't know the answer, asking your opinion!)
"Maybe they let you declare some subgraph as 'dynamic' to avoid static optimizations?" What you just described is Tensorflow Eager and why it has some performance issues (but more flexibility!). XLA makes some pretty strong assumptions and I don't think that should change. Tensorflow's ability to automatically generate good automatically parallelized production code stems from the restrictions it has imposed. So I wouldn't even try for a "one true AD to rule them all" since making things more flexible will reduce the amount of compiler optimizations that can be automatically performed.
To get the more flexible form, you really would want to do it in a way that uses a full programming language's IR as its target. I think trying to use a fully dynamic programming language IR directly (Python, R, etc.) directly would be pretty insane because it would be hard to enforce rules and get performance. So some language that has a front end over an optimizing compiler (LLVM) would probably make the most sense. Zygote and Diffractor uses Julia's IR, but there are other ways to do this as well. Enzyme (https://github.com/wsmoses/Enzyme.jl) uses the LLVM IR directly for doing source-to-source translations. Using some dialect of LLVM (provided by MLIR) might be an interesting place to write a more ML-focused flexible AD system. Swift for Tensorflow used the Swift IR. This mindset starts to show why those tools were chosen.
Makes sense. I don't use TF Eager, but I do use Jax, and Jax lets you arbitrarily compose JITed and non-JITed code, which made me think that might be a viable pattern. I guess I wondered if there might be something like "nonstatic_jit(foo)" that would do "julia style" compiling on function foo, in addition to "jit(foo)" that compiles foo to optimized XLA ops. Probably impractical. Thanks.
I am a python developer who has dabbled with Julia but it never stuck for me.
I think Julia was built by academics for other academics running innovative high performance computing tasks. It excels at the intersection of 1) big data, so speed is important, and 2) innovative code, so you can't just use someone else's C package. Indeed, Julia's biggest successful applications outside academica closely resemble an academic HPC project (eg Pumas). I think it will continue to have success in that niche. And that's not a small niche! Maybe it's enough to support a billion dollar company.
But most of us in industry are not in that niche. Most companies are not dealing with truly big data, on our scale, it is cheaper to expand the cluster than it is to rewrite everything in Julia. Most who ARE dealing with truly big data, do not need innovative code; basic summary statistics and logistic regression will be good enough, or maybe some cloud provider's prepackaged turn key neural nets scale out training system if they want to do something fancy.
I think for Julia to have an impact outside of academia (and academia-like things in industry) it will need to develop killer app packages. The next PyTorch needs to be written in Julia. Will that happen? Maybe! I hope so! The world would be better off with more cool data science packages.
But I think the sales pitch of "it's like Pandas and scikit but faster!" is not going to win many converts. So is Jax, Numba, Dask, Ray, Pachyderm, and the many other attempts within the Python community of scaling and speeding Python, that require much less work and expense on my part for the same outcome.
Again, congrats to the team, I will continue to follow Julia closely, and I'm excited to see what innovative capabilities come out of the Julia ecosystem for data scientists like me.
> The next PyTorch needs to be written in Julia. Will that happen? Maybe! I hope so!
The Two Language Problem makes this more likely than one might think. Those high level python packages that plaster over python's many mediocrities have to be written and maintained by someone, and while extremistan bears the brunt of the pain and has done a remarkable job shielding mediocristan, it's extremistan that gets to decide which language to use for the next killer app.
Of course, python has more inertia than god, so it won't go quietly.
I think this is a good deciding factor. Not just for “big data”. In my experience, with its combination of flexibility and raw speed, Julia makes implementing new algorithms (from scratch) a breezy experience. The strong research/academic presence in the community also helps towards encouraging decent Julia libraries for a lot of cutting edge work.
So if you are working in an area where that could make a significant difference, it’s an excellent reason to use Julia.
> will need to develop killer app packages. The next PyTorch needs to be written in Julia. Will that happen?
If enough cutting-edge work happens in Julia, it’s likely that a few great tools/platforms will emerge from that. We’re already seeing that in the scientific modeling ecosystem (as an example), with Differential Equations infrastructure and PumasAI.
There is another important niche I am particularly excited about: programming language research geeks and lisp geeks. The pervasive multiple-dispatch in Julia provides such a beautiful way to architecture a complicated piece of code.
> The pervasive multiple-dispatch in Julia provides such a beautiful way to architecture a complicated piece of code.
On the other hand it makes function discovery almost impossible [1]. Combined with the way how 'using' keyword exports a predefined subset of functions, this makes the language doomed from larger adoption outside of academia at least as long as there is no superb autocompletion and IDE support.
This may seem more about performance (than IDE development) but Shuhei is one of the driving contributors behind developing the capabilities to use compiler capabilities for IDE integration -- and indeed JET.jl contains the kernel of a number of these capabilities.
Great summary. I've worked with scientists that love Julia and more power to them. As a software engineer, there are still rough edges in productionizing Julia (yes, I know there are a few examples of large scale production code). As soon as you take Julia out of notebooks and try to build moderately complex apps with it, you realize how much you miss Python. Having used Julia for last 4 years and having to maintain that code in production environment, I am strongly convinced that Julia has a niche but it is not going to be a contestant to Python/Java/C++ depending on the use case. Which really is a shame - I want one goddamn language to rule them all. I want that and tried to give a fair chance to Julia.
There is too much to talk about and I’d want to give an objective impression with examples in a blog post, but one of the major grips I have is how little information Julia provides you with stack traces. Debugging production problems with absolutely zero clue of what/where the problem might be is one of the most frustrating aspects. I’ve spent so many hours debugging Julia using print statements. Debugger is janky, Juno/Atom support is not very good. Nothing feels polished and robust. Dependencies break all the time. We are stuck with Julia 1.2 and too much of an undertaking to go to latest version. Package management is an absolute disaster - this is the case with Python but Julia is worse. Julia Registry has many issues (compat errors). Testing and mocking is also underwhelming. I think startup times have improved but still not very developer-friendly. Sorry not an objective answer you’re looking for. There also things that Python has such as excellent web-dev ecosystem and system libs that are missing in Julia. Python has everything. Want to generate a QR code image? Easy in Python. Want to create PDF reports with custom ttf fonts? A breeze in Python.
This is the first time I’ve heard somebody say that Julia’s package management is worse than Python’s! For most people who have spent years grappling with the succession of unofficial attempts to supply Python with something like package management, including virtual environments, etc., and the resulting dependency hell, Julia’s integrated and sophisticated package management (even with its own special REPL mode) is refreshing. I don’t doubt your experiences are real, but suspect you have just had really bad luck.
1.2 is pretty ancient. Current, or even recent, versions of Julia have a fraction of the startup time (https://lwn.net/Articles/856819/). Package management has been refined further, as well.
I’m a tortured soul, my opinions might be biased and I hope things improve with Julia.
We absolutely cannot upgrade Julia version right now, dozens of repos full of complicated scientific code. Management doesn’t care as far as it barely runs. I don’t think it’s fair to blame Julia for it but it just shows how much more it needs to go. That should be looked at as a positive thing.
I have one more complain to Julia community - please don’t be too defensive. Accept and embrace criticisms and turn that into a propellant for improving the language. I’ve seen a cult-like behavior in Julia community that is borderline toxic. Sorry, it’s the truth and needs to be said. Speed isn’t everything and people are magnetized by the benchmarks on the Julia website, especially non-software engineers.
>I’ve seen a cult-like behavior in Julia community that is borderline toxic. Sorry, it’s the truth and needs to be said. Speed isn’t everything and people are magnetized by the benchmarks on the Julia website, especially non-software engineers.
I think all languages have this dynamic...I've seen it with python and R. To some extent it's fed by what we perceive as criticisms from people defending their favorite incumbent language with arguments that aren't at all informed- such as a focus on speed and how numba achieves parity there.
In the same vein, I and many Julia users are enthusiastic precisely because of thing other than speed, such as the type system, abstractions, differentiability and other things that make programming more joyful, fluid and productive.
Agree though, that we could always improve on acceptance of criticism.
Agreed and thanks for being charitable for the Julia community. It’s an interesting thing to balance: push and market the language, navigate haters and naysayers while also deeply respecting feedback and criticisms. We can all do better.
Well you’re right about the libraries—Python really does have everything (although sometimes those libraries are not great quality, but at least they mostly work). But Julia makes it easy to use Python libraries with Pycall. And there are big things that just don’t exist yet in the Julia world, such as a web framework. I recently tried to create a web project using Genie.jl, which advertises itself as a ready-for-prime-time web framework for Julia, and I gave up after a few days. It’s just not in the same universe as something like Django, plus it’s barely documented.
I really don't understand your point about package management.
Instead of Pip, virtualenv, conda, etc etc there's one package manager that resolves and installs native and binary dependencies, ensures reproducibility with human readable project files, is accessible from the REPL etc.
You can get an entire GPU based DL stack up and running on a windows machine in under 30 min, with a few keystrokes and no special containers or anything like that. Don't even have to install cuda toolkit. It's a dream, and I've heard the same from recent python converts
A lot of problems are fixable with time and money. Maybe the Series A will help!
But some problems might be related to Julia's design choices. One thing I really missed in Julia is Object Oriented Programming as a first class paradigm. (Yes I know you can cobble together an approximation with structs.)
OOP gets a lot of hate these days. Mostly deserved. But in some large complex projects it's absolutely the right abstraction. I've used OOP in several Python projects. Most of the big Python DS/ML packages use OOP.
Maybe you think PyTorch, SciKit, etc are all wrong, and would be better off with a more functional style. I know it's fashionable in some circles to make fun of the "model.fit(X,Y); model.predict(new_data)" style of programming but it works well for us busy professionals just trying to ship features.
I don't think Julia is wrong for enforcing a more functional style. It probably makes it easier for the compiler to generate great performance.
But Python has achieved its success because of its philosophy of being the "second best tool for every job" and that requires a more pragmatic, multiparadigm approach.
How is "model.fit(X,Y)" better than "fit!(model,X,Y)"?
Julia is object oriented in a broad sense, it just uses multiple dispatch which is strictly more expressive than single dispatch, so doesn't make sense to have dot notation for calling methods because types don't own methods.
For giving up some facility in function discover, you get speed, composability, generic code...and a net gain in usability because you can have one array abstraction for GPUs, CPUs etc etc, which is just an instance of having common verbs across the ecosystem (enabled by MD). Instead of everyone having their own table type or stats package, you have Tables.jl or Statsbase.jl that packages can plug into and extend without the issues inherent in subclassing, monkeypatching etc.
This is a much better, more powerful and pleasant experience
Closing the gap in Method discovery will simply require a language feature with tooling integration, where you write the types and then tab to get the functions. There's already an open issue/PR for this
I don't miss OOP in Julia but I do feel there need to be more ways to abstract things than multiple dispatch and structs. One thing I do miss is interfaces, which can group functions for a common purpose. I understand it may be not feasible in a dynamic language, but hopefully there will be something above functions as an abstraction mechanism.
Interfaces are not currently defined in the language but socially if you are lucky the package whose abstract type you subtype has a generic test you can call with your new implementation but that is not the standard.
I'll give you my two cents, recognizing that I very well might just be ignorant about Julia and multiple dispatch, and if so please continue to educate me.
Consider if we want to run many different types of models. Logistic regression, gradient boosting, NNs, etc. We want the ability to easily plug in any type of model into our existing code base. That's why model.fit(X,Y) is attractive. I just need to change "model = LogisticRegressionModel" to "model = GradientBoostingModel" and the rest of the code should still Just Work. This is a big part of SciKit's appeal.
But all these different models have very different training loops. So with "fit!(model,X,Y)" I need to make sure I am calling the compatible "fit" function that corresponds to my model type.
You might now say "Ah! Multiple dispatch handles this for you. The 'fit' function can detect the type of its 'model' argument and dispatch execution to the right training loop sub function." And I suppose that's theoretically correct. But in practice I think it's worse.
It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model." (They don't have to implement it, but they do have to import the right one.) The developer of "fit" should not be responsible for handling every possible "model" type. You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.
So is it possible to do the same thing with "fit!(model,X,Y)"? Yes of course it is. It's possible to do anything with any turing complete language. The point is, which system provides the best developer ergonomics via the right abstractions? I would argue, in many cases, including this one, it's useful to be able to bundle functions and state, even if that is in theory "less flexible" than pure functions, because sometimes programming is easier with less flexibility.
>It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model." (They don't have to implement it, but they do have to import the right one.) The developer of "fit" should not be responsible for handling every possible "model" type. You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.
It's really the same thing as python, just better...I don't see the distinction you are drawing.
In python you have a base class with default behavior. You can subclass that and inherit or override.
Julia has abstract types with interfaces...instead of relying on implementation details like fields, you provide functions so that more types of models can work even if they don't have that one specific field. Otherwise everything is the same where it counts,- you can compose, inherit and override. Even better, you can work with multiple models and types of data, inheriting where you see fit.
I don't see any benefit to python's restrictions here, either in ease of use or in expressiveness.
For all intents and purposes it's a strict superset.
Even better, you can use macros and traits to group different type trees together.
>It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model.
>You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.
It's the same in python, either you inherit Fit or you can override it. What's the difference with Julia?
Except in julia all types and functions have a multiple dispatch, parametric type and possible trait lattice of things you can override, customize and compose, so that even if the model author has to override fit, they can do it using small composable building blocks.
I agree you can achieve same benefits with Macros. Indeed, I see that MLJ, Julia's attempt at a SciKit type project, makes extensive use of Macros. But I personally think macros are an antipattern. In large projects, they can introduce subtle bugs. Especially if you're using multiple modules that are each editing your code before compile time and that don't know about each other. I know others in Julia community agree that Macros are dangerous.
I think abstract types are a brittle solution. The "can of worms" I alluded to is something like this: Library TensorFlow implements model "nn" and Library PyTorch also implements model "nn" and they both want to override "fit" to handle the new type "nn"... Good luck combining them in the same codebase. This problem is less pronounced in OOP where each development team controls their own method. Julia devs can solve this by having every developer of every "fit" function and every developer of every "model" struct agree beforehand on a common abstraction, but that's an expensive, brittle solution that hurts innovation velocity.
I think the closest I can do in Julia via pure structs is for the developer to define and expose their preferred fit function as a variable in the struct, something like "fit = model['fit_function']; fit(model,X,Y)" but that introduces a boilerplate tax with every method I want to call (fit, predict, score, cross validate, hyperpameter search, etc). (EDIT: indeed, I think this is pretty much what MLJ is doing, having each model developer expose a struct with a "fit" and "predict" function, and using the @load macro to automate the above boilerplate to put the right version of "fit" into global state when you @load each model... but as described above, I don't like macro magic like this.)
If (and only if, I have not looked at our hypothetical model and fit GF, but for the sake of argument, I will assume that it does) "fit" specialises on "model" will a "mode = AnOtherModel" cause "fit(model, x, y)" to be exquivalet to Python's "model.fit(x, y)". If you need to provide a custom "fit" method, you do so by providing a method specialised on AnOtherModel to the "fit" GF.
At no point is there a macro involved.
As for the "module a, model nn" and "module b, model nn", I would naively assume that they actually are different models, and therefore something specialising on "a.nn" will not get dispatched to when you pass a "b.nn".
Disclaimer: I don't actually know Julia, at all. But I have written substantial amounts of CLOS code (and Python, but I like CL and CLOS better).
I know nothing about Lisp, so at the risk of talking past eachother....
I never said macros were required. I said implementing this type of code without OOP required more boilerplate, and MLJ uses macros to reduce that boilerplate.
As I understand module imports in Julia: Each module developer exports a list of publicly facing objects. Obviously "fit" and "model" would be among them. If you import two modules that both export a new "nn" subtype of shared parent type "model", and both extend "fit" and "predict" and etc to accept their own subtype "nn", then you have to manually specify which module you are referring to every time you call nn, or fit, or predict, or whatever. Is that wrong? If I just import PyTorch, import TensorFlow, and then call "mymodel = TensorFlow.nn; fit(mymodel, mydata)" then Julia doesn't know that the "fit" I am calling is the TensorFlow implementation and not the PyTorch implementation; what if I had WANTED to use module A's "fit" on module B's model, and they intentionally adopted the same abstract type system to enable this interoperability? So instead I have to write "mymodel = TensorFlow.nn; TensorFlow.fit(mymodel, mydata); TensorFlow.predict(mymodel, mynewdata)". Obviously the extra typing is mildly annoying but the bigger problem is potentially introducing bugs by mismatching modules, and the developer's cognitive overload of having to keep track of modules. Python style OOP is a more elegant solution to the namespace problem and results in more readable, maintainble code, at least in my opinion. Anyways, maybe Julia has a more elegant solution I'm not aware of, if so I'd love to hear it.
In Julia it will just dispatch to the correct function.
In other words, one package would define `fit(mymodel::TensorFlowModel)` and the other would define `fit(mymodel:PyTorchModel)`, and then when you call `fit` it'll just dispatch to the appropriate one depending on the type of `mymodel`.
This dispatch-oriented style also allows a shocking degree of composability, e.g. [1], where a lot of packages will just work together, such that you could for example just use the equivalent of PyTorch or TensorFlow on the equivalent of (e.g.) NumPy arrays without having to convert anything.
If you mean "what about the case where both packages just call their model type `Model`", while I've never run into that, the worst case scenario is just that you have to fall back to the Python style explicit Module.function usage (which was always allowed anyways...). And if you if you don't like names being exported, you can always just `import` a package instead of `using` it:
help?> import
search: import
import
import Foo will load the module or package Foo. Names from the imported Foo module can
be accessed with dot syntax (e.g. Foo.foo to access the name foo). See the manual
section about modules for details.
Do you have an example of a case where you ran into this in Julia with two packages that you wanted to use together? If the packages are still actively developed, I suspect the developers would be interested to resolve the situation to allow interop.
Never made it that far. This was a feature I use all the time in Python ML development (both consuming open source packages via an OOP interface and also writing in-house model classes) that I consider essential for my productivity and that Julia was missing.
<edit>I'm also nervous of relying on the recourse of asking package maintainers to edit their variable names to improve compatibility with the random third package I want to use; maybe the culture is different in Julia but in Python that's a good way to get laughed out of the issue tracker :) </edit>
If I were you I would maybe ask people like @systemvoltage who took the plunge and wrote a big project in Julia only to find they had trouble maintaining the project. Maybe one reason he can't upgrade without breaking everything is because of namespace collisions amongst his many dependencies? If not that, it's something like that.
I know in Julia I can just precede every function call and object instantiation with "modulename." and solve the namespace problem that way. What I want to do instead, what I do in python, is bind one namespace of function methods to each object, so that as I code, I don't have to remember which module each each object came from. That is the appeal to me of "model.fit" over "module.fit(model)".
EDIT: This is not some "shave a few seconds off coding time" quality of life issue. This is a mission critical requirements in many enterprise contexts.
Scenario A: Model Development Team trains a model, serializes it, and sends the serialized model object to Production. You want Production to have to lookup which software Package the Model Development Team used for each model, just so they know which "predict" function to call? No, "predict" should just be packaged with the model object as a method, along with "retrain", "diagnose", etc.
Scenario B: I want to fit a dozen different types of models, from multiple packages, on the same data, to evaluate how they each do, and build build some sort of ensemble meta model. So I need to loop over each model object in my complicated ensembling logic. You want me to also programmatically keep track of which module each model comes from?
These are important, happens-every-day use cases in the enterprise.
I think the best solution for this in Julia is to package the model state with all the "method" functions in one struct. Again, this is what MLJ does. This is the closest Julia gets to OOP. But then you either need a lot of boilerplate code, or a Macro to unpack it all for you behind the scenes.
hmm, but doesn't it just mean that ppl should extend the same `fit!` method rather the define their own?
The bigger issue for production in my experience is about packaging the right model with the right version. I don't think anyone has to do `module1.fit` everywhere, since `fit` would've likely come from the same source.
> since `fit` would've likely come from the same source
No.
As described in the great-great-great-great-great-great-great-great-great-parent comment, the problem I am describing is that of trying to combine models from multiple software authors. You may not have that problem. It may not be a common problem among Julia's academic user base. I do have that problem.
Just to be clear, I like Julia, and think it has advantages over Python. I'm writing all this as someone who is cheering for Julia to break out of its HPC niche. Thanks.
Thanks so much for taking the time to outline your thoughts...I share the same goals and input from industry experience like this very valuable. This has spawned some discussion on the Julia slack about how best to target your usecase.
Can I trouble you to make a post either on discourse or on the slack? I'd really like this to get in front of the broader julia community and core devs, and you're the best person to do that. Maybe there's a solution of which I'm unaware....or there could be some design discussions to come out of this.
Always happy to help. Especially the last day or so - I've been waiting on some long training loops so it's been a pleasant diversion.
To be 100% honest with you, there's pretty much 0% chance of me adopting Julia in the next 12 months. I evaluated it before embarking on my current project, but ended up going with Python, and now I have several thousand lines of Python code that work fine, and I'm not going to rewrite it all in the near future. At some point in the medium term, I'll re-evaluate Julia. But until then, I don't want to lead anyone on any wild goose chase. Even if you solved this problem, and all my problems, I'm just not in a position to switch right now. So for that reason I think I'll hold off on issuing a community wide call for help. But I'm cautiously optimistic that at some point in the future I'll be writing Julia professionally.
A lot of this is also probably cultural rather than language features. The first thing they teach any Python data science boot camp initiate is: never "from numpy import *", always "import numpy as np" and yet in Julia "using" appears more common than "import"...
I also wouldn't read too much into my one example. It was initially meant just as an illustrative point, but somehow I was so bad at explaining it that it took tons of comments for me to get my minor point across. I do think the MLJ guys are properly on the right track, and that should work fine for most people who don't mind Macros. Maybe I'm in the minority in hating Macros.
The more commonly cited issues around compile time caching, tooling, etc. are boring to list off yet again, but probably the right areas of focus for the community, in terms of both impact and ease of implementation.
More generally, I really do think you're better off talking to people like @systemvoltage, who have actually given Julia more of a chance than I have. If I worked for Julia Computing I'd be reaching out to him and begging to get his brain dump on all the challenges he faced. In any business, it's always easier to reduce churn and learn from your (unhappy) customers, then it is to convert new prospects, whether that business is programming languages or investment banking.
Composability is one of the things the Julia community generally Takes Seriouslyᵀᴹ so definitely don't hesitate to ask if there are two packages that don't play as nicely as you would like!
I'm a bit confused still though why you say it's a "missing" feature, given that as we discussed above, there is absolutely nothing to stop anyone who wants to use the "Python OOP" style of namespacing in Julia from doing so? Most of us don't seem to find it necessary or to prefer it personally, but that doesn't restrict anyone else from choosing it.
Yes indeed. While Julia does generally eschew the "model.fit(X,Y); model.predict(new_data)" style, it's not because it's functional, it's because it's dispatch-oriented, which is arguably a superset of class-based OO, and even perhaps arguably closer to Alan Kay's claimed original vision for OO than modern class-based OO is.
Gosh, that's really old and it's not even an LTS version. I fear many of your woes stem from that. Julia 1.6 has huge improvements, and the community has rallied around VS Code now that Atom seems to be dying.
This matches with my experience with an old version of Julia too. But lets not focus on the negative aspects here. People have their different sets of priorities.
Did PyTorch increase or decrease demand for ML developers?
Did Ruby on Rails increase or decrease demand for web developers?
Did C increase or decrease demand for systems programmers?
Automation and abstraction is a complement, not a substitute.
If CoPilot works and we get to focus on higher level goals and developer throughout increases, it will be good for developers and good for the world. I hope CoPilot succeeds.
Mirrors my experience. Walled garden “we handle everything for you” tools are good for simple use cases. One you need to go outside the garden, even a little bit, it becomes more trouble than it’s worth. For non-trivial projects, I prefer docker + pip.
Here's an example on why a lock file is still very important when using Docker:
Let's say it's June 19th and your project exists on GitHub and you have a requirements.txt file with only `Flask==2.0.1` in it.
Now let's fast forward to October 19th and you clone the project and run a docker-compose build. You'll get Flask 2.0.1 but you might get a drastically newer version of Werkzeug, Jinja2, itsdangerous and Click because all of those dependencies are defined by Flask with the >= operator.
Suddenly you have a Docker image that's much different than what it was in June and the only thing that changed is time. This becomes a problem because now it could potentially mean running different versions in dev, CI and production.
This has bitten me in the past a few times with Werkzeug and also the Vine library with Celery where it pulled in newer versions that had backwards incompatible changes that broke my app. Things worked in the past but didn't work months in the future even when none of my top level dependencies changed.
A lock file fixes this issue and it's still necessary with Docker.
I've solved it in a slightly different way using pip directly by keeping top level deps in requirements.txt, freezing out a requirements-lock.txt file and referencing it with pip3 install using the -c flag. There's example repos at https://github.com/nickjj/docker-flask-example and https://github.com/nickjj/docker-django-example that demonstrate how it's done. It's not a 100% fool proof solution like Poetry but it works well enough where I haven't had a single issue since I started using this pattern and it mostly solves the problem.
It is, but with the strategy above a new lock file will get generated if the requirements.txt file is newer than the lock file, so if you change 1 dependency you might get newer unexpected dependencies. This is just the limitation of how pip works without building a whole new layer of dependency tracking in (which I guess is why Poetry and similar tools exists). Fortunately in practice I'm happy with the pip solution because it's a few line shell script and hasn't been a problem yet. The important part of having the lock file is there for reproduceable builds today and in the future.
the problem poetry.lock solves that requirements.txt doesn't is that if one of the dependencies you use has an unrestrictive version in their requirements.txt you can end up with unexpected upgrades in the dependencies of your dependencies. if you are only ever developing against a prebuilt docker container and that exact container is what gets shipped to prod then you won't have this issue, but if you have a CI system anywhere along the line that rebuilds the container you can still be bit by this.
I pip install foo, and foo depends on bar. I pip freeze > lock.txt. My lock file has foo v1 and bar v1, right? Later bar upgrades to v2. I then try to rebuild the container image from lock.txt. My pip freeze lock file will still keep me on foo v1 and bar v1, even though foo has unpinned dependency on bar and bar has new version.
Is pip freeze not solving this scenario? Or is poetry solving a different scenario?
Not trying to flame war, just not sure I’m grokking.
1) ETH bulls say that Solana is not really decentralized. It achieves scalability by limiting the network to only a handful of validators; why not just use AWS at that point? Why are they wrong?
2) ETH bulls say that security of a proof-of-stake is driven by total coin market cap; the more cap, the more expensive an attack. This makes ETH more secure than Solana. ETH has an unbreakable first mover advantage for this reason, they say.
3) ETH bulls say that ETH plus Polygon (or similar level two solution) is just as scalable as solana, while retaining the above two advantages of ETH. Why is that wrong?
PoS security driven by market cap is like saying that the richer the top 1% is and the more they’ll be incentivized to protect their funds. It’s not necessarily wrong, but it throws out a lot of other factors out the window. For example, who is this top 1% and how much do they care about the crypto.
Honest question: How is that different from proof of work, where those with significant resources (1%, a nation, etc) can spend significantly on mining equipment?
Whatever equipment you buy will lose it's value quickly, in 5+ years, it's probably worthless. And that's ignoring all the money you have to spend on electricity, non-stop and increasingly. In PoW, if miners slow down, they'll become irrelevant.
Even if you manage to acquire a 51% hash rate for example (which is extremely difficult), it'll be very difficult to keep it over a long enough period of time.
PoS is fundamentally broken in my opinion, it literally says "rich gets richer" and wealthy stakers will get higher (absolute) rewards which they don't have any incentive to sell since they didn't spend any energy to get it. That's even ignoring all the custodians which have large quantities of the said crypto, so they can just keep the rewards to themselves or keep a sizeable % of it.
One of the motivations for the DAO hack hardfork btw was that the attacker would hold significant power under a PoS system.
You can participate in mining pools, but realistically speaking unless you have very powerful hardware, you're better off just buying Bitcoin outright.
Same with PoS, you can stake with a pool if you have less than 32eth, but your returns on that are not going to be comparable to what large holders get (same percentage, but very tiny in absolute numbers).
You're arguing that "small" (<$80K) amounts of money aren't worth investing even at a very high APR. This is pretty contrary to most investment advice.
I'm not, even small amounts should be invested of course, I'm just pointing out that rich benefit the most in either scenario.
Also worth pointing out, high APRs aren't guaranteed and the network fees are supposed to be significantly decreased post EIP-1559. Also, for those who don't meet 32 eth threshold, there's good chance they are not even remotely close to that threshold.
The difference is that PoW is permissionless and PoS is permissioned system. Another difference is that in PoS you pay once and get benefits forever, in PoW you must actively choose to spend energy mining. One more difference is that once you become a staker, your power in the system only grows, in PoW anybody can start mining and dilute your power.
PoS is simply worse from every angle and it’s also not cheaper because MR=MC.
PoS is permissioned system because if you want to become a validator you must convince another (potential) validator to give up part of his stake.
> MC in PoS is mainly interest costs
no, it's also costs of fighting (via politics, marketing and hacking) for that initial pre-mined stake.
it's not cheaper than PoW and has worse security properties due to all sorts of attacks - long range, grinding, etc. ultimately it's flawed because unlike in PoW there is no universally objective measure of geniuneness of a chain (the "work" in PoW).
PoS is effectively permissionless given the number of parties that have the stakeable asset and are willing to sell it.
>>no, it's also costs of fighting (via politics, marketing and hacking) for that initial pre-mined stake.
There is no fight if there was a transition from PoW to PoS, and thus no premine, or if the premine was distributed via an open crowdsale, with revenue allocated to a non-profit foundation.
Your analysis makes too many tenuous assumptions to push one side of the debate.
> PoS is effectively permissionless given the number of parties that have the stakeable asset and are willing to sell it.
the market doesn't have enough liquidity for you to gain a meaningful stake and those that organized the pre-mine scam will always remain in control. or they will sell you the stake for exorbitant price and perform a long-range attack against you because they still hold the original keys.
Eth is going through transition and yet it’s the largest pre-mine scam in existence.
Crowd sales are just pre-mines.
Non-profit is just pre-mine beneficiary that people will fight for control over.
And you conveniently ignored all the other problems with PoS: long range and grinding attacks, no ability to reduce power of malicious staker, no universal objective measure for which chain is genuine so you have to rely on third parties, etc.
>>the market doesn't have enough liquidity for you to gain a meaningful stake
This is unsupported. There are billions of dollars worth of ETH sold every day. If you intended to hold what you bought, the liquidity would gradually go down as you took what was bought off the market, but you could certainly acquire a significant share of the stake.
>>Non-profit is just pre-mine beneficiary that people will fight for control over.
That assumes zero altuistic oversight from ETH stakeholders at large deterring attempts to corrupt the grant issuing process, which is not a sensible assumption.
>>And you conveniently ignored all the other problems with PoS
Long-range attack is addressed with dependence on weak subjectivity:
> There are billions of dollars worth of ETH sold every day
not nearly enough to purchase a meaningful stake that could protect you from premine scammers that launched the project. not to mention - you'd be giving up your money for their benefit. double rekt.
> zero altuistic oversight from ETH stakeholders at large deterring attempts to corrupt the grant issuing process, which is not a sensible assumption
ah, nice, a system that simply relies on altruistic motives of premine scammers that will be in control and largest beneficiaries of those staking grants. what could possibly go wrong.
> Long-range attack is addressed with dependence on weak subjectivity: ... The rest of your criticisms have similarly been addressed in state of the art Proof of Stake protocols
it's not addressed, it's just partly waved away and partly obfuscated in a non-solutions like slashing or checkpointing.
pos still doesn't work, pow is the only known decentralized and trustless consensus reaching protocol.
>>not nearly enough to purchase a meaningful stake that could protect you from premine scammers that launched the project. not to mention - you'd be giving up your money for their benefit. double rekt.
There is no need for any protection. Proof of stake doesn't enable stakers to attack non-stakers. Nor would stakers have any incentive to.
Moreover, there was absolutely no scam in the premine. It was publicly announced, and the majority of it was distributed via a programmatic crowdsale.
This characterization of yours is simply an emotional attack.
>>you'd be giving up your money for their benefit. double rekt.
Same with any currency. You provide something of value to acquire some currency. This applies to dealing with early adopters of other currencies, early investors in companies, etc.
This is simply a bad-faith criticism of proof-of-stake that is equally applicable to anything else, unless you make the tortured argument that a publicly announced crowdsale and dev grant is somehow a "scam", and therefore there is some distinct quality about buying currency from those who acquired their stake through a premine over acquiring it through some other method.
>>ah, nice, a system that simply relies on altruistic motives of premine scammers
How can any one can take ETH's critics seriously when you make blatantly libelous accusations that participating in an open premine crowd makes someone a scammer.
> Proof of stake doesn't enable stakers to attack non-stakers
yes it does. it allows stakers to prevent non-stakers from becoming stakers. all rewards go to stakers. rich get richer even faster.
> there was absolutely no scam in the premine. It was publicly announced, and the majority of it was distributed via a programmatic crowdsale.
public announcement means nothing if participation is permissioned. there was 12% blatant premine and 60% so called "pre-sale", of which undisclosed amount went to scammers that organized it and didn't have to pay anything.
> applies to dealing with early adopters of other currencies
this doesn't apply to BTC which literally anybody could mine without asking approval and permission. ETH is just another scam.
> dev grant is somehow a "scam"
of course it is.
> open premine crowd
well, at least you used the right word to describe it - premine. any crypto premine is a scam by definition. some of those scams just manage to bamboozle enough people to stay afloat longer and get a chance to scam even more.
good job shifting conversation away from discussing PoS flaws into complaining about randoms on internet being rude to scammers.
>>yes it does. it allows stakers to prevent non-stakers from becoming stakers. all rewards go to stakers. rich get richer even faster.
No it doesn't. As I already explained, there is no practical way holders can collude to force all holders of the currency to not sell. There will always be significant liquidity for any currency that has as wide a distribution of holders that Ethereum does.
>>public announcement means nothing if participation is permissioned.
The crowdsale was not permissioned. It was completely programmatic.
>>there was 12% blatant premine
Which was publicly disclosed compensation for the developers who created Ethereum, as well as an allocation for grants to further develop Ethereum.
>>and 60% so called "pre-sale", of which undisclosed amount went to scammers that organized it and didn't have to pay anything.
This allegation of the pre-sale being a scam is totally unsubstantiated. It's irresponsible character assassination.
>>randoms on internet being rude to scammers.
The credibility of avowed critics of Ethereum, who make totally unsubstantiated allegations of the organizers of Ethereum's crowdsale of being scammers, is relevant to these discussions.
Moreover, your criticism is not relevant to PoS. It's specifically critical of Ethereum, since Ethereum had a premine and crowdsale. PoS doesn't have to have either. So once again, your analysis seems entirely biased and agenda-driven.
Basically in proof of work you have no idea who is participating at any moment. For example, the NSA could be running a blockchain in parallel and if they have more power create a fork. With proof of stake you are always aware of the participants in the system and a fork will only happen it the richest decide to fork. But why would they? In Bitcoin the most powerful miner might have no bitcoin and no incentive to keep the network safe.
At the same time, because of this, you don’t really get finality in Bitcoin. You get some assurance that your transaction is “confirmed” after a number of block but you’re never really sure. In proof of stake you can have consensus protocols with true finality, meaning that there’ll be no forks for sure if a threshold of participants remained honest.
My understanding is, proof of stake is not more secure than proof of work. Reason for PoS are efficiency, speed of transaction, lower gas fees, less environmental impact, etc. (Of course there can be secure PoS, insecure PoS, secure PoW, insecure PoW…)
Polygon basically checkpoints a spreadsheet onto the eth chain on an interval, but there is no way to guarantee that there weren't any shenanigans between the checkpoints. This is a big improvement over say BSC which does no such thing, but it isn't the security of a true eth L2. Real L2's can prove that their output to L1 is legitimate either through zero knowledge cryptographic proofs (loopring, zksync) or through a game theory fraud check (optimism, arbitrum).
PoS is more secure in general, also PoS and BFT-based cryptocurrencies in periods of network partitions will rather come to a stop instead of allowing safety to be violated (double spending).
Once they stop, they don't start again without external intervention (so you're back to The DAO when it comes to which validators are not Byzantine). PoS is a bet that 1/3 of the staked tokens never, ever fall into the hands of Byzantine actors -- not by purchase, not by theft, and not by DeFi smart contract hacks. That's not a bet I would take.
And how, exactly, will the "network conditions stabilize" if over 1/3 of the votes are malicious, and thus able to prevent the honest voters from ever agreeing on anything ever again? Are you betting that the attacker will just get bored and walk away?
Also, what a confusing choice of words. A distributed system is BFT (or not BFT) regardless of whether or not the underlying message broadcast medium is synchronous/asynchronous, or reliable/unreliable. The "network conditions" being "stable" have no bearing on the voters' ability to reach agreement -- that's solely a function of whether or not f or fewer votes are malicious out of 3f + 1 votes.
I was talking about network conditions, not a threshold of malicious participants (in which case yeah you will have liveness issues).
Your second paragraph is false also. Different BFT systems have different assumptions. Some work in asynchronous settings, some work in semi-synchronous settings, etc.
You should consider rereading Leslie Lamport's original paper. BFT is a property of corrupt votes, not the network. You keep trying to make it about the network. Like, if you want to have a conversation about how the network can influence the system's fault tolerance, you should instead consider the network topology -- as in, which routes between honest nodes include corrupt nodes. This is also considered by the paper, since corrupt nodes can censor or rewrite messages, and thus influence how many corrupt nodes the system can truly tolerate, given a network topology. But in no case does message delay give a BFT system's node an excuse to make forward progress without first verifying that at least 2f+1 replicas agree with its decision. Even voting on a view change to remove a presumed dead node requires a 2f+1 vote.
I wouldn't say PoS security has to be driven by the rich, but by whoever is staking, which people could be staking small like lets say 1 ETH total. The security is derived simply by the threat of having your staked ETH (however much) slashed away.
Nothing is slashed if attacker doesn’t publish the alternative chain until they have successful attack.
PoS is absolutely driven by rich and helps rich get richer faster than everybody else. If I get 10% stake - I get 10% of any future issuance, meaning my stake can never go below 10%, so my power in the system never dilutes even though I literally don’t have to do anything anymore. Miners in PoW have to participate in mining and their power can get diluted by anyone by simply getting more hardware online.
There isn’t a cap on validators so you can’t buy up a fixed percentage of the network. More people can always join. You will be diluted over time unless you choose to reinvest(same as mining).
As we reduce the hardware costs and energy usage costs it becomes easier to participate in the network (especially via pools, same as mining but much much cheaper).
Being able to run a validator on a solar powered raspberry pi is a great improvement to making participation in the network accessible.
We should see the exact opposite of what you suggest, anyone who wants to participate not having energy or hardware restrictions should make it less Matthew-effect-like.
PoS increases both the cost of a direct attack on the network as reorganised/51s are more expensive to perform with slashing mechanisms in place, and also removes the threats of supply line disruption by either nation states or cartels forming to control the flow of the hardware.
> There isn’t a cap on validators so you can’t buy up a fixed percentage of the network
you can during a pre-sale or pre-mine event
> More people can always join
joininng as validator means convincing another validator to reduce their stake (sell it to you), which is a form of permission.
> You will be diluted over time unless you choose to reinvest
you can't be diluted if you don't sell you stake and continue staking. that's just by definition how PoS works.
> PoS increases both the cost of a direct attack on the network as reorganised/51s are more expensive to perform with slashing mechanisms in place, and also removes the threats of supply line disruption by either nation states or cartels forming to control the flow of the hardware.
nope, literally none of it is true.
slashing mechanisms only obfuscate the attack, they don't make it more expensive. in fact they reduce security by virtue of piling more and more rules that require more and more code, which inevitably contains bugs.
threats of supply chain attacks are much less scary than threats of long range attacks from hacked / overtaken private keys of early / current validators.
producing more hardware to counter an attack might be expensive and early iterations of hardware can be inefficient, but at least nobody can stop you from producing it. as i've already explained - if somebody gets a stake in pos system, there is nothing you can do to reduce it.
pos simply doesn't work. it's been known a decade before pow and was just never considered seriously because it's not trustless and permissionless.
I think people care about decentralization only to an extent. They really just want to it to be decentralized enough that the SEC or CFPB won't shut it down. I call this 'plausible decentralization'.
If you try to run an unlicensed exchange without KYC on AWS, you will get shut down pretty quick.
I think the government will look at Solana and say its too complicated to shut down.
That only works as long as the government stays clueless. If Solana or Angorand or whatever gets big enough and the government employs some tech-savvy people to examine it, they'll figure out that there is a subset of blessed validators out there that could be legally compelled to cease and desist and that this would cripple the network.
That wouldn't really work for Bitcoin or other PoW coins. They could shut down big mines and new ones would pop up. Of course they could instead invest a lot in both seizing mines and building their own and 51% the network... that's always a risk and any large enough nation state could do it.
It’s the same for Bitcoin btw. The government can just forbid any VASP to accept bitcoin and to authorize on/off ramp and trades involving bitcoin. ISPs could also block miners and bitcoin websites. The coin is censorship resistant… to a certain extent. Maybe there’ll always be someone running it, and there’ll be a black market for on/off ramp, but it will severely cripple the adoption.
'it will severely cripple the adoption', I'm not sure it's so straightforward as that, if large numbers of governments take an authoritarian stance on independent global currencies is there not a possibility that, while impacting network efficiency, it might underpin a new found desire and need for said independent global currency and instead drive adoption of currencies that have this resilience?
I think it’s worth than that for bitcoin as bitcoin relies on a connected network for security. The risk of forks is too high without that. For other cryptocurrencies based on BFT at least the network would come to a stop and safety wouldn’t be violated (no double spending).
I don't think they really care, especially once they are invested. They may care enough to do minimum research before investing in a token, but once they are invested it is like watching football. Algorand claims to be decentralized, but relies on gatekept relays and a centralized domain. If you bring this up, it is like a cult where how dare you question anything about the coin we've invested in.
A big risk I'm concerned about is say some whistleblower posts some classified material to the Solana Blockchain, that the US government wants removed. Does Solana have the ability to roll back the chain or remove specific pieces of information from it?
If so, where is the line of what they will / won't remove? I fear we get into a Facebook like situation where nobody is happy where that line is drawn. Could legitimate projects be shut down because a government has deemed them illegal because they aren't complying with regulations?
> Does Solana have the ability to roll back the chain or remove specific pieces of information from it?
Well potentially if enough validators forked the chain before the material was added and were able to build a longer chain than the one with the classified material, but even then it would be pointless as the previous chain still exists and will have been distributed to every node.
The question is, why would they want to remove it? Solana is a global blockchain, not a US government entity. If they were to abide by US government requests, it would be only fair to also abide by the requests of other governments - but what if North Korea wanted information removed? As you said, where do we draw the line over what is a "valid" request?
And more importantly, who draws the line? There is no single Solana entity, it is a group of validators who would decide this. Achieving consensus on something like this would never happen, especially among a group of people who are trying to be resistant to government censorship.
> If you try to run an unlicensed exchange without KYC on AWS, you will get shut down pretty quick.
The next step in cryptocurrency technology is to decentralize the exchanges themselves. Ideally it should be impossible to shut them down, regulate them or even understand what's going on. Governments will either give up or become tyrants in the process of fighting increasingly subversive technology. We'll find their limits.
Almost there.. each network has its own decentralized exchange where you can swap and buy tokens. We are just missing contracts for derivatives such as futures and leverage trading.
#3 polygon is a centralized scaling solution. The question should be about decentralized options like rollups. Some examples are arbitrum, optimism, and zksync.
Polygon right now has capped the number of validators at 100, and the protocol atop which it is built (Tendermint/“Peppermint”) only remains fast and cheap if the “active set” of validators remains small. Polygon team have said they will introduce an “auction” mechanism for new validators to join the active set, but one would anticipate that will just favor validators with large stakes (i.e. the existing validator active set). Cosmos/ATOM is the sort of keeper of Tendermint and responsible for the reference implementation and they too cap the validator active set at 125 or 130. Practically speaking none of these L2s are very decentralized. They are fast and cheap which is what most people want but practically pretty centralized...(i.e. project founders and early operators are totally capable of launching 50% or 2/3rds attacks...they probably just don’t want to, and the small “active set” of validators is likely to mostly stay the same). A genuinely decentralized PoS L2 (i.e. can accommodate very large number of validators) that is also fast and cheap is something I’d love to see
You didn't answer any of the questions, especially regarding decentralization. If there's going too be only a handful of validators, then it's as good as a centralized app hosted on AWS. Then how is it facilitating "DeFi", when there's no "De"?
I’ve read a lot about Solana and have considered trying to get a Validator up on their testnet.
I think the concern about centralization of the Solana chain is a real one. I don’t know what to say about the coin distribution in general and frankly that is my biggest concern with Sol. But in regards to validator centralization, I wonder how many validators would be required to assuage these concerns. Thousands? Tens of thousands? Currently there are 632 validators on mainnet Solana with the largest stake holding I see close to 5% (1). Compare that to the ETH mining pools where the top two pools almost have a majority of mining hashrate (2). I’m not trying to make a “whatabout” argument, just pointing out the relative decentralization in comparison to current Eth network.
With regards to the eth2 network, the comparison is a bit more straightforward. If we do a calc of the market cap / #ofvalidators, Sol has about 16M$ market cap per validator. I see 172,920 Eth2 validator deposits (3) so if the Merge happened right now, each validator would represent about 1.7M$ (=300B / 172920). So in this highly oversimplified model, Eth2 has about 10x as many validators as Sol.
This of course means nothing about real world outcomes as you could imagine large staking pools forming and the situation looking more like the BTC or ETH PoW chains with a few large groups dominating either Sol or Eth2
Anyway, for me, that’s actually kind of reassuring. Solana wants to push out more validators but block times are so fast that growing the network has some real technical challenges. If they can overcome those challenges, they’ll be reasonably decentralized with regards to the number of validators, at least compared to ETH.
- ability for the system to run entirely in people's home. Anything that requires datacenters can be easily regulated.
- distributed holdings with no entity holding more than few percent at most.
Solana fails on both, most egregiously on the latter. Only 4.3% of coins were even offered in a public sale. The idea that defi can run on a chain controlled entirely by few VCs is ridiculous. Even Libra would be preferable with their Swiss based Libra Association.
1) Decentralization is a spectrum, BTC -> ETH -> SOL, AWS will be really difficult to get some blockchain properties: permissionless & composability.
For smart contract platforms, imo some level of decentralization can be sacrificed for higher throughput, read more here: https://haseebq.com/why-decentralization-isnt-as-important-a....
2) ETH will be more secure than Solana! Solana requires both higher bandwidth and a beefy machine[1] to run a validator node.
https://solanabeach.io/validators
3) Polygon is both less scalable & controlled by a multi-sig(less decentralized) which is also similar to BSC. Polygon is based on EVM, Solana is BPF/Rust based with some unique set of optimizations like Sealevel[2] which enables parallel processing.
