Rust bindings improvements

[domenukk]

These bindings from unicornafl allow us to

• use a data ptr (if this one is needed is debatable, what do you think?)
• have closure that don't live for'static anymore -> easier to capture things in
• have non-boxed closures (perf++, hopefully)
• no_std (potentially smaller)
• no longer need to cast regs to i32 manually

[domenukk]

@wtdcode got rid of the additional API

[wtdcode]

Merged, thanks. ;)

[domenukk]

@wtdcode one thing, in case you autogenerate the register ids, make sure you add

impl From<RegisterARM> for i32 {
    fn from(r: RegisterARM) -> Self {
        r as i32
    }
}

to each file :)

[wtdcode]

@wtdcode one thing, in case you autogenerate the register ids, make sure you add

impl From<RegisterARM> for i32 {
    fn from(r: RegisterARM) -> Self {
        r as i32
    }
}

to each file :)

Rust bindings constants are not generated automatically unfortunately and that's on my TODO plan.

[bet4it]

There are two types of mem callback:

unicorn/include/unicorn/unicorn.h

Lines 283 to 319 in 52f90cd

	/*
	Callback function for hooking memory (READ, WRITE & FETCH)
	@type: this memory is being READ, or WRITE
	@address: address where the code is being executed
	@size: size of data being read or written
	@value: value of data being written to memory, or irrelevant if type = READ.
	@user_data: user data passed to tracing APIs
	*/
	typedef void (*uc_cb_hookmem_t)(uc_engine *uc, uc_mem_type type,
	uint64_t address, int size, int64_t value, void *user_data);
	/*
	Callback function for handling invalid memory access events (UNMAPPED and
	PROT events)
	@type: this memory is being READ, or WRITE
	@address: address where the code is being executed
	@size: size of data being read or written
	@value: value of data being written to memory, or irrelevant if type = READ.
	@user_data: user data passed to tracing APIs
	@return: return true to continue, or false to stop program (due to invalid memory).
	NOTE: returning true to continue execution will only work if the accessed
	memory is made accessible with the correct permissions during the hook.
	In the event of a UC_MEM_READ_UNMAPPED or UC_MEM_WRITE_UNMAPPED callback,
	the memory should be uc_mem_map()-ed with the correct permissions, and the
	instruction will then read or write to the address as it was supposed to.
	In the event of a UC_MEM_FETCH_UNMAPPED callback, the memory can be mapped
	in as executable, in which case execution will resume from the fetched address.
	The instruction pointer may be written to in order to change where execution resumes,
	but the fetch must succeed if execution is to resume.
	*/
	typedef bool (*uc_cb_eventmem_t)(uc_engine *uc, uc_mem_type type,
	uint64_t address, int size, int64_t value, void *user_data);

uc_cb_eventmem_t has a return value but uc_cb_hookmem_t doesn't.
You may need to add a new add_mem_invalid_hook function.

uc_cb_hookinsn_invalid_t and uc_cb_insn_in_t also have a return value:

unicorn/include/unicorn/unicorn.h

Lines 190 to 206 in 52f90cd

	/*
	Callback function for tracing invalid instructions
	@user_data: user data passed to tracing APIs.
	@return: return true to continue, or false to stop program (due to invalid instruction).
	*/
	typedef bool (*uc_cb_hookinsn_invalid_t)(uc_engine *uc, void *user_data);
	/*
	Callback function for tracing IN instruction of X86
	@port: port number
	@size: data size (1/2/4) to be read from this port
	@user_data: user data passed to tracing APIs.
	*/
	typedef uint32_t (*uc_cb_insn_in_t)(uc_engine *uc, uint32_t port, int size, void *user_data);

You may change them in this PR.

Originally posted by @bet4it in #1421 (comment)

mem_hook_proxy shouldn't always return a bool.

[domenukk]

@wtdcode one thing, in case you autogenerate the register ids, make sure you add

impl From<RegisterARM> for i32 {
    fn from(r: RegisterARM) -> Self {
        r as i32
    }
}

to each file :)

Rust bindings constants are not generated automatically unfortunately and that's on my TODO plan.

I had it running at one point, but probably needs some adaption to newer changes, take a look at:
https://github.com/AFLplusplus/unicornafl/blob/6c9ed25bfb8ff8650921fc27be74ba4e6b7282a4/bindings/const_generator.py

[domenukk]

There are two types of mem callback:

unicorn/include/unicorn/unicorn.h

Lines 283 to 319 in 52f90cd

	/*
	Callback function for hooking memory (READ, WRITE & FETCH)
	@type: this memory is being READ, or WRITE
	@address: address where the code is being executed
	@size: size of data being read or written
	@value: value of data being written to memory, or irrelevant if type = READ.
	@user_data: user data passed to tracing APIs
	*/
	typedef void (*uc_cb_hookmem_t)(uc_engine *uc, uc_mem_type type,
	uint64_t address, int size, int64_t value, void *user_data);
	/*
	Callback function for handling invalid memory access events (UNMAPPED and
	PROT events)
	@type: this memory is being READ, or WRITE
	@address: address where the code is being executed
	@size: size of data being read or written
	@value: value of data being written to memory, or irrelevant if type = READ.
	@user_data: user data passed to tracing APIs
	@return: return true to continue, or false to stop program (due to invalid memory).
	NOTE: returning true to continue execution will only work if the accessed
	memory is made accessible with the correct permissions during the hook.
	In the event of a UC_MEM_READ_UNMAPPED or UC_MEM_WRITE_UNMAPPED callback,
	the memory should be uc_mem_map()-ed with the correct permissions, and the
	instruction will then read or write to the address as it was supposed to.
	In the event of a UC_MEM_FETCH_UNMAPPED callback, the memory can be mapped
	in as executable, in which case execution will resume from the fetched address.
	The instruction pointer may be written to in order to change where execution resumes,
	but the fetch must succeed if execution is to resume.
	*/
	typedef bool (*uc_cb_eventmem_t)(uc_engine *uc, uc_mem_type type,
	uint64_t address, int size, int64_t value, void *user_data);

uc_cb_eventmem_t has a return value but uc_cb_hookmem_t doesn't.
You may need to add a new add_mem_invalid_hook function.

uc_cb_hookinsn_invalid_t and uc_cb_insn_in_t also have a return value:

unicorn/include/unicorn/unicorn.h

Lines 190 to 206 in 52f90cd

	/*
	Callback function for tracing invalid instructions
	@user_data: user data passed to tracing APIs.
	@return: return true to continue, or false to stop program (due to invalid instruction).
	*/
	typedef bool (*uc_cb_hookinsn_invalid_t)(uc_engine *uc, void *user_data);
	/*
	Callback function for tracing IN instruction of X86
	@port: port number
	@size: data size (1/2/4) to be read from this port
	@user_data: user data passed to tracing APIs.
	*/
	typedef uint32_t (*uc_cb_insn_in_t)(uc_engine *uc, uint32_t port, int size, void *user_data);

You may change them in this PR.

Originally posted by @bet4it in #1421 (comment)

mem_hook_proxy shouldn't always return a bool.

Ah, missed that. Will take a look at it later and open another pr (unless someone else wants to)

[domenukk]

@bet4it what do you recommend? Drop the bool return again? Add a hook?

[bet4it]

Add add_mem_invalid_hook and add_insn_in_invalid_hook function.