Cyril Jover and Mathieu Muller of Unity Technologies discuss their photogrammetry workflow at Siggraph 2017. Here they present the technology behind the de-lighting tool and how to get started with photogrammetry. You can see how they take advantage of the benefits of Artomatix (About 23 minutes into the presentation). For a detailed look at the range of features in the ArtEngine tool from Artomatix, head to the product overview page: https://artomatix.com/create-overview.php
By Dr. Eric Risser
A.I. seems to be in the news a lot these days. I am often then asked why do I think A.I. is gaining so much momentum compared to other emerging technologies? I believe there are several factors that contribute to this:
First, the internet + the “internet of things” has produced incomprehensible amounts of data. A.I. is the only tool capable of extracting a signal out of all that noise, for this reason, industry and government are both deeply invested in A.I.
Second, Deep Learning, which is the new buzzword for Neural Networks, a subcategory of A.I., has recently transitioned from a 50-year long science fair project to a breakthrough technology that is achieving state of the art results rivaling humans in several fields. A big part of this recent success is due to the availability of big data sets, which thanks again to the internet, can be used for training, and GPU’s becoming general purpose enough that they are now capable of running Neural Networks orders of magnitude larger and faster than was previously possible. One example is Nvidia, who has put considerable resources over the past five years into turning graphics cards into Neural Network supercomputers. It is, in fact, thanks to all of this, that things like self-driving cars, which would have been science fiction five years ago, are now a reality.
Third, I believe the most important reason why A.I. is taking center stage, is due to its wide application in so many fields. Like computers, or the internet, A.I. is not a technology that is been designed around a single purpose or application, it is a general-purpose “enabler” technology that can enhance and improve almost anything from biotech to architecture. Chances are that cancer and cold fusion are going to be solved with the help of an Artificial Intelligence in the future. On that note, Artomatix is applying A.I. to the creative industry, a field that has never utilized it before, and we are seeing some amazing things as a result!
This might make you ask if I am asserting that A.I. is mainstream now and has wide applications? To this I say, exactly! A.I. is not really Terminators, or some kind of Will Smith style iRobots. A.I. is in every aspect of your digital life already. For example – right now facebook’s neural network facial recognition is so good that if someone takes a picture of you and puts it in their newsfeed, Facebook knows you are in that picture, if you are not already friends, then Facebook can recommend you. Another example, your search engine queries are constantly being monitored by an Artificial Intelligence. They can act as virtual doctors and alert you to serious health conditions you might have such as a rare form of cancer if you happen to search for the right combination of specific symptoms… helpful or creepy, I leave it to you to decide.
This is why A.I. is now widespread for analytical tasks, but creativity and imagination is something that we like to think is still uniquely human. So how is it A.I. can be applied to the 3D art creation process?
Traditionally computers have been very bad at simulating creativity. The reason is that the common computer, i.e. the Turing machine, follows a very different computational model from the neural networks inherent to life on Earth. In many ways, computers are far superior to brains. The average CPU can do around three billion calculations per second, while the human brain maxes out at around 30. Computers are extremely good at onboarding, storing and processing huge amounts of data very precisely, whereas brains are relatively slow to learn new information, it’s often stored in a flawed and incomplete manner and our calculations are far from precise. That said, for many tasks including being creative, the human brain vastly outperforms the Turing Machine. The reason is parallelism. Computers only have two to eight concurrent processes running in parallel (i.e. they can only think about a few things at the same time), whereas the human brain is a massively parallel computer with millions of threads all running together. The brain also constantly rewires itself, making new connections and breaking old ones. Memories and ideas can meld together to create new experiences in our heads. It’s our imprecision that makes us so good at coming up with new ideas and that’s why computers have never really been applicable to the creative fields… until now.
Now you are probably asking yourself why would an artist want an A.I. to compete with them? I think this is a common misconception that A.I. will compete with or replace humans. The key is, A.I. can program itself to an extent, but at its core, there is going to be some fundamental metric that it uses to measure success, and this cannot ever change. This is actually true for any intelligence, including our own. For life on earth, our fundamental metric of success is survival and reproduction (hence why we are so afraid of competition). When it comes to A.I. however, we can start over from scratch. At Artomatix our A.I. measures its success by how well it helps our users. Its happiness is derived from your satisfaction!
There is actually a lot for artists to gain by embracing A.I. in their workflow. To start with, we can draw a lot of parallels with the evolution of the word processor. Originally people used typewriters to create documents, when PCs became commonplace in the early 80’s the typewriter was one of the first things to go, Computers gave people the ability to correct mistakes and make digital backups of their work to edit/re-print later. The creative industries also moved to computers for the same reason. Artists abandoned their paint brushes and drafting tables long ago for Wacom tablets and Photoshop.
That said, word processing has come a long way since the 80’s while art software has not. Spelling and grammar checkers are commonplace now. Autocomplete can often predict what we are going to write and finish text messages for us. Speech recognition has gotten so good recently that documents can now be dictated rather than typed manually.
The real value of computers is their ability to do work that only humans could do previously. Applying Artificial Intelligence to artistic tasks is kind of the same. In fact, we are already starting to see these “smart tools” emerge, such as Photoshop’s Content-Aware Fill, which is to images what auto-complete is to word processing. I think initially we’re going to see a lot of these “smart tools” pop up, which will help fix/auto-complete the really tedious and not very creative effort that goes into making 3D content. After that, we will see tools that can learn content and style from an artist and mass produce it. That is what Artomatix is currently focusing on.
In the long term though A.I. will likely enhance human creativity in more meaningful ways and lead to fundamentally new ways of being artistic. For example, imagine a system where you wear a helmet that monitors your brainwaves and you sit in front of a screen that shows you what you are thinking and feeling. Being able to see this on the screen will automatically trigger new thoughts and ideas which will, in turn, be fed back into the system. This would make a creativity feedback loop where ideas are quickly generated and refined with very little effort.
Pete McNally is a Senior Designer/3D Generalist with over 14 years experience in the Irish games industry. He works with an Emmy award-winning tech firm, and he has also worked with a twice Oscar-nominated animation studio. We were really impressed with some work that he did recently using Artomatix as part of our Alpha group, which he subsequently published on his blog: https://petemcnally.com/
Here’s what Pete had to say.
“I had a partial scan that hadn’t resolved well it was wet rock on a very sunny day so large areas of detail were missed or were blurry. I baked out what I had in 3DSMAX, diffuse, normals, AO, height and a shadow map used to help with manual de-lighting in Photoshop. I ran these textures through Artomatix for seam removal and it tiled them quite nicely. After some tweaking, I applied the material to a sphere in Toolbag 3 and tested out some lighting environments, before applying the same material to the inside of a curved cylinder and tiling appropriately, to make the cavernous environment you see below. Not bad for a single material!”
Here’s the post that Pete published when he gave Artomatix a run through for texture mutation and seam removal. https://petemcnally.com/2018/03/02/seamless-materials-from-3d-scans-in-artomatix/
And here is his further exploration of working with Artomatix and partial scans. : https://petemcnally.com/2018/03/08/the-60-minute-cave-part-ii/
- The Top Three Games of 2018 as Voted by Artomatix
- NEWS: COMMENT ON STATEMENT FROM AN TAOISEACH AND MINISTER HUMPHREYS IN RELATION TO €75M DISRUPTIVE TECHNOLOGIES INNOVATION FUND
- ZenEarth VR Uses AI to Help a Scan-based Photogrammetry Work-flow
- Direct on mesh synthesis of PBR materials now supported by ArtEngine from Artomatix
- Unity discusses Artomatix as part of their photogrammetry workflow.