Procedurally generated clouds

November 9, 2015 — For awhile now clouds have been on my mind, specifically, clouds for seamless cubemap planet textures. I have come up with a method that seems to produce passable results. Here are a couple examples:

The method begins with another method I used for generating cubemap terrain textures. That method involves sampling real terrain elevation data to programmatically create “brushes” on the fly which are then used to recursively paste and blend elevation patches onto a sphere. The key insight I had was that I could do exactly the same thing with with satellite cloud imagery — treat it as if it were elevation data, and paste and blend dynamically created “elevation” patches consisting of slightly processed and sampled grayscale cloud imagery — I didn’t even have to write any code, I just gave my existing program different data to chew on.

This by itself worked ok, but lacked the kind of “swirliness” that you see when you look at pictures of earth taken from space, as you can see below.

For this swirliness, I fell back to my other nifty program gaseous-giganticus, used for creating gas-giant planet textures. I noticed that the “swirliness” of planets like Jupiter is quite different from the swirliness of planets like Earth. I spent some time looking at this super cool real-time earth wind thing and some time playing with the noise-scale parameter of gaseous-giganticus to get a better handle on how to produce a given sort of swirliness. Here are some pictures showing the effect of various noise scale values in gaseous-giganticus.

Once I could produce some satisfactory looking grayscale cloud cubemaps, it was a simple matter to take a white image and use the cloud images as an alpha mask and composite it onto terrain images.

In my Space Nerds In Space github repository, I describe in some detail exactly how to do all this. See: How to Generated Earth-like Planet Textures.

~ by scaryreasoner on November 10, 2015.

10 Responses to “Procedurally generated clouds”

  1. hey mr steve.
    can i have your email to send you some question about io requests in linux kernel.
    recently i read your post about life of an io request.
    i have some question which cant find their answer by searching, and i believe you can answer them. also these questions are very important and significant for my work and research.
    i would be grateful if you five me your email.
    my academic email has been add.

  2. I don’t know what “my academic email has been add” means — I don’t see it. Just ask your question.

  3. ok, i meant when i want to post a comment here, there is a required filed named email! which i wrote my email. (academic email = my college email).
    i read your post a bout the life of an IO request in linux kernel, that post is awesome and very informative and helpful.
    now i how some questions:
    1- if system has several disks, in which layer or section or code, kernel code determines which disks this io request corresponds to?
    2- how one can find out that current io request is a read request or a write request?
    3- which address this io request wants to read from or write to?
    4- how much is the io request size?
    5- which user app. sent this io request?
    i would be grateful if you answer my questions.
    thank a lot.

  4. hi,
    is there any answer to my question ?

    • Hi, I replied to your query, below. (just adding this additional reply here in case it triggers an email notification to you.)

  5. Hi, Ata, Sorry, been out of town for a while. Let me try to answer your questions.

    Question 1: The device will be determined essentially at the time open() is called. (Note “the device” does not have to be an actual hardware device, it might be a software raid device that ends up triggering subsequent i/o’s to underlying devices.) You can use the “stat” command (or stat() system call) to examine a file, for example if you “stat” /etc/passwd:

    $ stat /etc/passwd
    File: ‘/etc/passwd’
    Size: 1832 Blocks: 8 IO Block: 4096 regular file
    Device: 802h/2050d Inode: 20709610 Links: 1
    Access: (0644/-rw-r–r–) Uid: ( 0/ root) Gid: ( 0/ root)
    Access: 2015-12-31 16:28:00.522691400 -0800
    Modify: 2015-12-16 20:34:20.277303579 -0800
    Change: 2015-12-16 20:34:20.281303579 -0800
    Birth: –

    Note the “Device:” line with the device number 802 (in hex) or 2050 (decimal). Note also:

    # cat /proc/partitions
    major minor #blocks name

    8 0 500107608 sda
    8 1 524288 sda1
    8 2 487099392 sda2
    8 3 12482560 sda3

    So the “8” and the “02” correspond to major and minor device numbers, so /etc/passwd resides on sda2.

    open() is going to create a file descriptor (an integer) which leads to an entry in the file descriptor table, which is going to ultimately have this major and minor device number in there somewhere (I know that’s a bit hand-wavy but I don’t have time to dig into the actual code right now.)

    So the association with a device happens at the time open() is called.

    Question 2: I am not sure what is meant by “the current” i/o request. There can be (and on a busy system generally are) many i/o requests concurrently in progress at any given time on any given device. There may be concurrent requests in progress from many processes or even from a single process. There’s not really any such thing as “the current” i/o request. If by i/o request, you mean a “struct request” as defined in include/linux/blkdev.h, that contains “q”, which is a pointer to a struct request_queue. Look in struct request_queue (in include/linux/blkdev.h) there is a struct backing_dev_info, which contains a pointer to a struct device, (see include/linux/device.h) which contains a dev_t (defined in include/linux/types.h) which is a typedef for a __kernel_dev_t, which is ultimately a u32, which contains the packed major and minor device numbers. Note also (see below) that a struct bio (many of which are contained within struct request) contains (iirc) a struct block_device, which also contains a dev_t, so that’s another way.)

    Question 3: which address an i/o request wants to read or write to. First of all, do you mean virtual or bus address? The devices only know about bus addresses. The CPU only knows about virtual addresses. Secondly, a single request may target many addresses in a scatter gather list. A struct request contains a pointer to a struct bio. See include/linux/blk_types.h. That contains another pointer to another bio, forming a linked list of bios. Each bio in the chain contains a struct bio_vec pointer. This is an “io vector” (here, “vector” is being used in the mathematical sense meaning “array”, essentially.) The b_vcnt member tells how many entries i the bio_vec array. The bio_vec contains a pointer to a struct page and an offset and a length. The struct page will get you to a page in the virtual memory system (I won’t go into the details of what’s in struct page) and the offset and length tell which bytes within the page the i/o is targeting. So each struct request has a linked list of bios, and each bio has an array of bio_vecs, and each bio_vec has a page, offset and length. Something along those lines.

    Question 4: io request size. Add up the lengths of all the bio_vecs in the request.

    Question 5: Which user app sent the request: Note it is possible that *no* user app sent the request, for example, some activity may be initiated directly by the kernel, or by the kernel on behalf of user processes in general, but in an asynchronous, kind of detached way. In general, once an i/o request has gotten into the block layer, the kernel doesn’t really care which user app submitted the request, and only cares once the i/o completes (successfully or not) at which point it needs to inform the process(es) waiting — and for that it doesn’t really even care, except to be able to find the right completion structure(s) to awaken. So in general, you will not find riding along with each i/o request some process ID or thread ID or anything like that. That part, I cover towards the end of the post:

    wake_up_process(dio->waiter);

    Note also it has been more than a year since I’ve done any kernel work, and I was never what you could call an expert on any of this, so it is quite possible I missed some details and got some things wrong, so do not take this as being completely correct. It’s just my (possibly mistaken) understanding right now, having taken a quick look at the code after more than a year of not having done anything with this kind of code.

    Hope that helps.

    • Thank a lot for your comprehensive and informative answer.
      in fact in the duration you were offline and didnt answer my question i dove into linux kernel and inserted some printk in some section of vfs layer, block io layer and scsi layer and monitored some info about an io request.
      now according to your answer, would you give me an example for question 3 and 4? that is give me a C code that uses a printk to print an io request address and size?
      i would be grateful.

    • ok 🙂 so i reduce it. for io request size i wrote this code:
      unsigned short io_size = 0;
      for(int i=0; ibi_vcnt; i++){
      io_size += bio->bio_vec->bv_len;
      }
      so i wanna you just give me something like that for io address.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: